[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:26:31 dist-test-slave-7vjv krb5kdc[1065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:26:31 dist-test-slave-7vjv krb5kdc[1065](info): set up 2 sockets
Aug 06 22:26:31 dist-test-slave-7vjv krb5kdc[1065](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:26:32.604740 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.425s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:26:32 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:26:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:26:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:26:32.808020 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38053
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:33.308368 1089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:33.308828 1089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:33.309094 1089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:33.318274 1089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:26:33.318459 1089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:33.318588 1089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:33.318733 1089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:26:33.318827 1089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:26:33.330070 1089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:33.331846 1089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:33.334002 1089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:33.353309 1099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:33.353399 1101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:33.353731 1098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:33.354882 1089 server_base.cc:1034] running on GCE node
I20260806 22:26:33.356014 1089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:33.357707 1089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:33.358948 1089 hybrid_clock.cc:648] HybridClock initialized: now 1786055193358872 us; error 87 us; skew 500 ppm
I20260806 22:26:33.359380 1089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:33 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055193, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:33.363495 1089 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:26:33.365271 1089 webserver.cc:460] Webserver started at http://127.1.8.126:37005/ using document root <none> and password file <none>
I20260806 22:26:33.366138 1089 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:33.366380 1089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:33.366748 1089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:33.370988 1089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3"
format_stamp: "Formatted at 2026-08-06 22:26:33 on dist-test-slave-7vjv"
server_key: "7ae72b39e6044348a8bff59b8967e0f8"
server_key_iv: "c6fb444653471e9cccff3d2dce6c25b5"
server_key_version: "encryptionkey@0"
I20260806 22:26:33.371795 1089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3"
format_stamp: "Formatted at 2026-08-06 22:26:33 on dist-test-slave-7vjv"
server_key: "7ae72b39e6044348a8bff59b8967e0f8"
server_key_iv: "c6fb444653471e9cccff3d2dce6c25b5"
server_key_version: "encryptionkey@0"
I20260806 22:26:33.378158 1089 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 22:26:33.383013 1109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:33.384640 1089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:26:33.384879 1089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3"
format_stamp: "Formatted at 2026-08-06 22:26:33 on dist-test-slave-7vjv"
server_key: "7ae72b39e6044348a8bff59b8967e0f8"
server_key_iv: "c6fb444653471e9cccff3d2dce6c25b5"
server_key_version: "encryptionkey@0"
I20260806 22:26:33.385124 1089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:33.422195 1089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:33.427522 1089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:33.427938 1089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:33.461127 1089 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38053
I20260806 22:26:33.461261 1194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38053 every 8 connection(s)
I20260806 22:26:33.462774 1089 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:26:33.467300 1195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:33.470667 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1089
I20260806 22:26:33.471087 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:26:33.473170 1057 external_mini_cluster.cc:1428] Setting key 50cd0113cc2e69628295dfb1a34dcad2
I20260806 22:26:33.481729 1195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3: Bootstrap starting.
I20260806 22:26:33.485430 1195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:33.486753 1195 log.cc:824] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:33.490849 1195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3: No bootstrap required, opened a new log
I20260806 22:26:33.501042 1195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } }
I20260806 22:26:33.501647 1195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:33.501922 1195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1c4b94ae824b0e9e33839d4bcb8dd3, State: Initialized, Role: FOLLOWER
I20260806 22:26:33.502679 1195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } }
I20260806 22:26:33.502995 1195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:26:33.503191 1195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:26:33.503417 1195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:33.506420 1195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } }
I20260806 22:26:33.507179 1195 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a1c4b94ae824b0e9e33839d4bcb8dd3; no voters:
Aug 06 22:26:33 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:33.508255 1195 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:26:33.508437 1204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:33.511197 1195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:26:33.511919 1204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 1a1c4b94ae824b0e9e33839d4bcb8dd3, State: Running, Role: LEADER
I20260806 22:26:33.512760 1204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } }
I20260806 22:26:33.522807 1208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1c4b94ae824b0e9e33839d4bcb8dd3. Latest consensus state: current_term: 1 leader_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } } }
I20260806 22:26:33.522854 1205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c4b94ae824b0e9e33839d4bcb8dd3" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38053 } } }
I20260806 22:26:33.523461 1208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:33.523479 1205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:33.527766 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:33.477153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.1.8.126:38053)
0806 22:26:33.478715 (+ 1562us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:33.478745 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:33.480211 (+ 1466us) server_negotiation.cc:407] Connection header received
0806 22:26:33.492240 (+ 12029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:33.492265 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:33.492637 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:33.493064 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:33.496447 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:33.498332 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:33.500382 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:33.501148 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:33.501546 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:33.510261 (+ 8715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:33.510276 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:33.510301 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:33.510369 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:33.516797 (+ 6428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:33.518252 (+ 1455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:33.518264 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:33.518275 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:33.518638 (+ 363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:33.519496 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:33.519505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:33.519518 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:33.520638 (+ 1120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:33.524869 (+ 4231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:33.525281 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 22:26:33.525819 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":1000,"threads_started":1}
I20260806 22:26:33.537912 1221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:26:33.545019 1221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:26:33.554611 1221 catalog_manager.cc:1347] Generated new cluster ID: 7745a4ad13664f44a25734195585fa6d
I20260806 22:26:33.554773 1221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:26:33.568007 1221 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:26:33.569630 1221 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:26:33.582851 1221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a1c4b94ae824b0e9e33839d4bcb8dd3: Generated new TSK 0
I20260806 22:26:33.584199 1221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:26:33.644285 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:34.173094 1238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:34.173388 1238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:34.173681 1238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:34.187319 1238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:34.187572 1238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:34.187893 1238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:26:34.200671 1238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:34.202131 1238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:34.203614 1238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:34.222437 1248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:34.222625 1249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:34.222630 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:34.224766 1238 server_base.cc:1034] running on GCE node
I20260806 22:26:34.225618 1238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:34.226892 1238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:34.228281 1238 hybrid_clock.cc:648] HybridClock initialized: now 1786055194228149 us; error 50 us; skew 500 ppm
I20260806 22:26:34.228808 1238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:34 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:34.233748 1238 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:26:34.236258 1238 webserver.cc:460] Webserver started at http://127.1.8.65:33425/ using document root <none> and password file <none>
I20260806 22:26:34.237152 1238 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:34.237378 1238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:34.237823 1238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:34.241412 1238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "ba0dcfe810a5426f9add1f3de2c42c10"
format_stamp: "Formatted at 2026-08-06 22:26:34 on dist-test-slave-7vjv"
server_key: "e1c55dd25eeee55b4f80b80f1ebb1e4a"
server_key_iv: "cce24ea7d06c7ef6e0ffe3189f87cab4"
server_key_version: "encryptionkey@0"
I20260806 22:26:34.242451 1238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "ba0dcfe810a5426f9add1f3de2c42c10"
format_stamp: "Formatted at 2026-08-06 22:26:34 on dist-test-slave-7vjv"
server_key: "e1c55dd25eeee55b4f80b80f1ebb1e4a"
server_key_iv: "cce24ea7d06c7ef6e0ffe3189f87cab4"
server_key_version: "encryptionkey@0"
I20260806 22:26:34.248960 1238 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 22:26:34.253679 1267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:34.255144 1238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260806 22:26:34.255379 1238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "ba0dcfe810a5426f9add1f3de2c42c10"
format_stamp: "Formatted at 2026-08-06 22:26:34 on dist-test-slave-7vjv"
server_key: "e1c55dd25eeee55b4f80b80f1ebb1e4a"
server_key_iv: "cce24ea7d06c7ef6e0ffe3189f87cab4"
server_key_version: "encryptionkey@0"
I20260806 22:26:34.255705 1238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:34.287726 1238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:34.293978 1238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:34.294516 1238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:34.296038 1238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:34.298127 1238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:34.298313 1238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:34.298584 1238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:34.298760 1238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:34.375154 1238 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:38315
I20260806 22:26:34.375377 1474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:38315 every 8 connection(s)
I20260806 22:26:34.377153 1238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:26:34.381125 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1238
I20260806 22:26:34.381400 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:26:34.382468 1057 external_mini_cluster.cc:1428] Setting key cbef77f874c4cf7165aa922534913460
Aug 06 22:26:34 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:34.407821 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:34.380771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44357 (local address 127.1.8.126:38053)
0806 22:26:34.381502 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:34.381506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:34.387834 (+ 6328us) server_negotiation.cc:407] Connection header received
0806 22:26:34.387955 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:34.387970 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:34.388135 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:34.388324 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:34.390751 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:34.391766 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:34.393865 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:34.394492 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:34.394640 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:34.399867 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:34.399872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:34.399876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:34.399908 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:34.403439 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:34.404720 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:34.404726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:34.404730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:34.404967 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:34.405643 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:34.405651 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:34.405657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:34.406106 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:34.406268 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:34.407020 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 22:26:34.407292 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":456,"threads_started":1}
I20260806 22:26:34.408779 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:34.381633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38053 (local address 127.1.8.65:44357)
0806 22:26:34.382709 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:34.382775 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:34.387206 (+ 4431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:34.388614 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:34.388624 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:34.389578 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:34.390527 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:34.390578 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:34.391945 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:34.391950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:34.393700 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:34.393713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:34.394867 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:34.394878 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:34.395163 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:34.396244 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:34.396266 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:34.399558 (+ 3292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:34.403612 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:34.403622 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:34.403640 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:34.404465 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:34.405163 (+ 698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:34.405167 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:34.405169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:34.405502 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:34.406245 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:34.406252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:34.406616 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 22:26:34.406852 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 22:26:34.407276 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":507,"threads_started":1}
I20260806 22:26:34.411091 1475 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38053
I20260806 22:26:34.411537 1475 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:34.413185 1475 heartbeater.cc:511] Master 127.1.8.126:38053 requested a full tablet report, sending...
I20260806 22:26:34.415795 1133 ts_manager.cc:194] Registered new tserver with Master: ba0dcfe810a5426f9add1f3de2c42c10 (127.1.8.65:38315)
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
I20260806 22:26:34.417721 1133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:44357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:26:34.449517 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:35.007719 1484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:35.008216 1484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:35.008406 1484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:35.017591 1484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:35.017736 1484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:35.017896 1484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:26:35.030786 1484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:35.032428 1484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:35.034260 1484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:35.052656 1493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:35.053750 1496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:35.053874 1494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:35.055250 1484 server_base.cc:1034] running on GCE node
I20260806 22:26:35.056070 1484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:35.057237 1484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:35.058531 1484 hybrid_clock.cc:648] HybridClock initialized: now 1786055195058409 us; error 59 us; skew 500 ppm
I20260806 22:26:35.058974 1484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:35 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055195, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:35.063320 1484 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:26:35.065241 1484 webserver.cc:460] Webserver started at http://127.1.8.66:37337/ using document root <none> and password file <none>
I20260806 22:26:35.066016 1484 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:35.066257 1484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:35.066613 1484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:35.070473 1484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "e5c694295170483586b36b34138f09a6"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "190d0bca7a08ddc8702ba8431d4c491e"
server_key_iv: "d767565140ce7fabd8fb42f2906f9cef"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.071462 1484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "e5c694295170483586b36b34138f09a6"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "190d0bca7a08ddc8702ba8431d4c491e"
server_key_iv: "d767565140ce7fabd8fb42f2906f9cef"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.078127 1484 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 22:26:35.082602 1505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:35.084036 1484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:26:35.084267 1484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "e5c694295170483586b36b34138f09a6"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "190d0bca7a08ddc8702ba8431d4c491e"
server_key_iv: "d767565140ce7fabd8fb42f2906f9cef"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.084553 1484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:35.110201 1484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:35.115552 1484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:35.115985 1484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:35.117493 1484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:35.119516 1484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:35.119748 1484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:35.120013 1484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:35.120196 1484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:35.198031 1484 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:45887
I20260806 22:26:35.198102 1692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:45887 every 8 connection(s)
I20260806 22:26:35.199622 1484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:26:35.208514 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1484
I20260806 22:26:35.208808 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:26:35.209553 1057 external_mini_cluster.cc:1428] Setting key 332721e05022f7e25a01826937666334
Aug 06 22:26:35 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055195, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:35.227490 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:35.203104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:58631 (local address 127.1.8.126:38053)
0806 22:26:35.203603 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:35.203609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:35.205020 (+ 1411us) server_negotiation.cc:407] Connection header received
0806 22:26:35.207469 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:35.207473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:35.207539 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:35.207787 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:35.210590 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:35.211832 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:35.213839 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:35.214967 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:35.215221 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:35.219803 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:35.219823 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:35.219830 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:35.219876 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:35.223548 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:35.224665 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:35.224669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:35.224671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:35.224830 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:35.225416 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:35.225419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:35.225421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:35.225734 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:35.225847 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:35.226937 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0806 22:26:35.227107 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":241,"threads_started":1}
I20260806 22:26:35.229708 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:35.203752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38053 (local address 127.1.8.66:58631)
0806 22:26:35.204685 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:35.204732 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:35.207196 (+ 2464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:35.208142 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:35.208156 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:35.209213 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:35.210401 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:35.210423 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:35.212030 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:35.212034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:35.213681 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:35.213694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:35.215314 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:35.215324 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:35.215614 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:35.216605 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:35.216630 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:35.219463 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:35.223699 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:35.223711 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:35.223730 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:35.224542 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:35.224971 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:35.224979 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:35.224985 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:35.225300 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:35.225885 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:35.225897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:35.226432 (+ 535us) client_negotiation.cc:769] Sending connection context
0806 22:26:35.226807 (+ 375us) client_negotiation.cc:240] Negotiation successful
0806 22:26:35.227626 (+ 819us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":482,"threads_started":1}
I20260806 22:26:35.231598 1693 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38053
I20260806 22:26:35.231961 1693 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:35.233057 1693 heartbeater.cc:511] Master 127.1.8.126:38053 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
I20260806 22:26:35.234831 1133 ts_manager.cc:194] Registered new tserver with Master: e5c694295170483586b36b34138f09a6 (127.1.8.66:45887)
I20260806 22:26:35.235726 1133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:58631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:26:35.268056 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:26:35.422192 1475 heartbeater.cc:503] Master 127.1.8.126:38053 was elected leader, sending a full tablet report...
W20260806 22:26:35.867501 1700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:35.867918 1700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:35.868173 1700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:35.877585 1700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:35.877818 1700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:35.878062 1700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:26:35.894381 1700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38053
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:35.896651 1700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:35.898293 1700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:35.916837 1710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:35.917699 1711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:35.918671 1713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:35.919271 1700 server_base.cc:1034] running on GCE node
I20260806 22:26:35.920217 1700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:35.921317 1700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:35.922647 1700 hybrid_clock.cc:648] HybridClock initialized: now 1786055195922562 us; error 66 us; skew 500 ppm
I20260806 22:26:35.923067 1700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:35 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055195, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:35.927472 1700 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:26:35.929608 1700 webserver.cc:460] Webserver started at http://127.1.8.67:46237/ using document root <none> and password file <none>
I20260806 22:26:35.930527 1700 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:35.930756 1700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:35.931187 1700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:35.935174 1700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "e58b9468a4b34dd5afb649de06559043"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "c8c4cadae08f0f674f167eba479fedb8"
server_key_iv: "47c44df1d977c03758cf2478abde6c4a"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.936224 1700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "e58b9468a4b34dd5afb649de06559043"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "c8c4cadae08f0f674f167eba479fedb8"
server_key_iv: "47c44df1d977c03758cf2478abde6c4a"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.942832 1700 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260806 22:26:35.947710 1726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:35.949115 1700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:35.949345 1700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "e58b9468a4b34dd5afb649de06559043"
format_stamp: "Formatted at 2026-08-06 22:26:35 on dist-test-slave-7vjv"
server_key: "c8c4cadae08f0f674f167eba479fedb8"
server_key_iv: "47c44df1d977c03758cf2478abde6c4a"
server_key_version: "encryptionkey@0"
I20260806 22:26:35.949656 1700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:35.976778 1700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:35.982283 1700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:35.982632 1700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:35.984110 1700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:35.986044 1700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:35.986200 1700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:35.986445 1700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:35.986590 1700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:36.070046 1700 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:37647
I20260806 22:26:36.070256 1907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:37647 every 8 connection(s)
I20260806 22:26:36.071686 1700 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:26:36.076377 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1700
I20260806 22:26:36.076681 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:26:36.077546 1057 external_mini_cluster.cc:1428] Setting key e2eee0f0caa5254d653c54906db5c792
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055195, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:36.105844 1914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.075559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:53591 (local address 127.1.8.126:38053)
0806 22:26:36.076628 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.076636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.078442 (+ 1806us) server_negotiation.cc:407] Connection header received
0806 22:26:36.082445 (+ 4003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.082451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.082573 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.082879 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.085400 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.086428 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.088458 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.089244 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.089364 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.098584 (+ 9220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.098589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.098593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.098629 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.102429 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.103175 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.103179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.103181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.103354 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.103816 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.103820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.103822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.104253 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.104373 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.105072 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.105294 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":652,"threads_started":1}
I20260806 22:26:36.107302 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.076470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38053 (local address 127.1.8.67:53591)
0806 22:26:36.078077 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:36.078139 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:36.082097 (+ 3958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:36.083432 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:36.083452 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:36.084378 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:36.085227 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:36.085249 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.086563 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.086567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:36.088327 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:36.088341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.089641 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.089649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:36.090022 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.094731 (+ 4709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:36.094758 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:36.098347 (+ 3589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:36.102576 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:36.102584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:36.102599 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:36.103053 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:36.103463 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:36.103466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:36.103468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:36.103709 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:36.104376 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:36.104382 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:36.104737 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 22:26:36.104975 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 22:26:36.105485 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187,"thread_start_us":825,"threads_started":1}
I20260806 22:26:36.110015 1908 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38053
I20260806 22:26:36.110448 1908 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:36.111572 1908 heartbeater.cc:511] Master 127.1.8.126:38053 requested a full tablet report, sending...
I20260806 22:26:36.113909 1133 ts_manager.cc:194] Registered new tserver with Master: e58b9468a4b34dd5afb649de06559043 (127.1.8.67:37647)
I20260806 22:26:36.115638 1133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:53591
I20260806 22:26:36.122277 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:36.182431 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.162747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.1.8.65:38315)
0806 22:26:36.163883 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.163896 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.164278 (+ 382us) server_negotiation.cc:407] Connection header received
0806 22:26:36.164553 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.164562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.164786 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.165021 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.166683 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.167872 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.169623 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.170360 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.170551 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.175669 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.175681 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.175698 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.175733 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.179144 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.179917 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.179924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.179927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.180207 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.180653 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.180658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.180661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.181057 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.181428 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.181722 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.182018 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":656,"threads_started":1}
W20260806 22:26:36.183758 1308 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46772
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260806 22:26:36.239800 1693 heartbeater.cc:503] Master 127.1.8.126:38053 was elected leader, sending a full tablet report...
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:36.256124 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.230546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.1.8.65:38315)
0806 22:26:36.230806 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.230813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.231153 (+ 340us) server_negotiation.cc:407] Connection header received
0806 22:26:36.231316 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.231320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.231415 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.231504 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.232947 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.233711 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.234998 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.235394 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.235491 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.250535 (+ 15044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.250542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.250545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.250586 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.253405 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.254114 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.254117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.254120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.254313 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.254666 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.254669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.254671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.254975 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.255235 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.255566 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.255749 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 22:26:36.257823 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1238
I20260806 22:26:36.274364 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.258901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.1.8.65:38315)
0806 22:26:36.259064 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.259068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.260333 (+ 1265us) server_negotiation.cc:407] Connection header received
0806 22:26:36.260510 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.260515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.260588 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.260798 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.262443 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.263184 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.264870 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.265402 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.265558 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.268382 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.268387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.268391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.268423 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.271170 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.272003 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.272009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.272013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.272234 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.272698 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.272704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.272707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.273017 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.273308 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.273690 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.273922 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:26:36.275559 1467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:36.392251 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1238
I20260806 22:26:36.418795 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1484
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:26:36.435485 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.420036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.1.8.66:45887)
0806 22:26:36.420825 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.420836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.420873 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:26:36.420953 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.420962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.421205 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.421537 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.422624 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.423628 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.425028 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.425554 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.425649 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.429575 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.429585 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.429597 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.429627 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.432591 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.433234 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.433247 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.433249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.433367 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.433764 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.433768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.433771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.434209 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.434525 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.434856 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.435145 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":397,"threads_started":1}
I20260806 22:26:36.436847 1687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:36.524298 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1484
I20260806 22:26:36.546412 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1700
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:26:36.563553 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:36.547630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.1.8.67:37647)
0806 22:26:36.548302 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:36.548310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:36.548342 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:26:36.548416 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:36.548422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:36.548658 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:36.548787 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:36.550025 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.551036 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.552659 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:36.553217 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:36.553318 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:36.558019 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:36.558031 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:36.558046 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:36.558093 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:36.560994 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.561618 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.561625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.561627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.561754 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:36.562073 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:36.562076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:36.562078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:36.562430 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:36.562655 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:36.562915 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:26:36.563119 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":308,"threads_started":1}
I20260806 22:26:36.565004 1898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:36.649430 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1700
I20260806 22:26:36.673570 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1089
I20260806 22:26:36.674878 1182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:36.778498 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1089
2026-08-06T22:26:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5665 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1945](info): set up 2 sockets
Aug 06 22:26:36 dist-test-slave-7vjv krb5kdc[1945](info): commencing operation
krb5kdc: starting...
W20260806 22:26:38.132483 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:26:38 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:26:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:26:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:26:38.348408 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34751
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:38.682404 1966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:38.682777 1966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:38.682958 1966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:38.690662 1966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:26:38.690907 1966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:38.691064 1966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:38.691215 1966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:26:38.691308 1966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:26:38.703016 1966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:38.704633 1966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:38.706238 1966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:38.721494 1976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:38.722059 1979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:38.722123 1977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:38.723584 1966 server_base.cc:1034] running on GCE node
I20260806 22:26:38.724555 1966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:38.726043 1966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:38.727308 1966 hybrid_clock.cc:648] HybridClock initialized: now 1786055198727252 us; error 39 us; skew 500 ppm
I20260806 22:26:38.727666 1966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:38 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:38.730970 1966 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:26:38.732605 1966 webserver.cc:460] Webserver started at http://127.1.8.126:35389/ using document root <none> and password file <none>
I20260806 22:26:38.733337 1966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:38.733498 1966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:38.733866 1966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:38.737007 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "85677bf0a2ca4e5dafe6422d94facdf7"
format_stamp: "Formatted at 2026-08-06 22:26:38 on dist-test-slave-7vjv"
server_key: "0e0708aed43bf457162fc329bd84be02"
server_key_iv: "6bb8c6af94ae635f42d20611a23eafae"
server_key_version: "encryptionkey@0"
I20260806 22:26:38.737789 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "85677bf0a2ca4e5dafe6422d94facdf7"
format_stamp: "Formatted at 2026-08-06 22:26:38 on dist-test-slave-7vjv"
server_key: "0e0708aed43bf457162fc329bd84be02"
server_key_iv: "6bb8c6af94ae635f42d20611a23eafae"
server_key_version: "encryptionkey@0"
I20260806 22:26:38.743064 1966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:26:38.746773 1988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:38.748126 1966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:26:38.748342 1966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "85677bf0a2ca4e5dafe6422d94facdf7"
format_stamp: "Formatted at 2026-08-06 22:26:38 on dist-test-slave-7vjv"
server_key: "0e0708aed43bf457162fc329bd84be02"
server_key_iv: "6bb8c6af94ae635f42d20611a23eafae"
server_key_version: "encryptionkey@0"
I20260806 22:26:38.748562 1966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:38.764896 1966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:38.769438 1966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:38.769761 1966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:38.788867 1966 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34751
I20260806 22:26:38.788866 2070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34751 every 8 connection(s)
I20260806 22:26:38.790333 1966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:26:38.794091 2071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:38.799309 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1966
I20260806 22:26:38.799562 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:26:38.800311 1057 external_mini_cluster.cc:1428] Setting key 242d2284fe11de7d3c05e90397ae9428
I20260806 22:26:38.805303 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7: Bootstrap starting.
I20260806 22:26:38.809720 2071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:38.811187 2071 log.cc:824] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:38.814785 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7: No bootstrap required, opened a new log
Aug 06 22:26:38 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:38.824668 2071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } }
I20260806 22:26:38.825111 2071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:38.825232 2071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85677bf0a2ca4e5dafe6422d94facdf7, State: Initialized, Role: FOLLOWER
I20260806 22:26:38.825954 2071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } }
I20260806 22:26:38.826227 2071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:26:38.826409 2071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:26:38.826606 2071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:38.827436 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:38.802361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.1.8.126:34751)
0806 22:26:38.803276 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:38.803288 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:38.803331 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:26:38.805095 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:38.805118 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:38.805530 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:38.805899 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:38.807867 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:38.810074 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:38.812302 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:38.813446 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:38.813635 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:38.819429 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:38.819436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:38.819450 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:38.819486 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:38.822693 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:38.823526 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:38.823533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:38.823539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:38.823706 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:38.824190 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:38.824199 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:38.824202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:38.824791 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:38.825007 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:38.825681 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 22:26:38.826170 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":688,"thread_start_us":445,"threads_started":1}
I20260806 22:26:38.829134 2071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } }
I20260806 22:26:38.829630 2071 leader_election.cc:302] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85677bf0a2ca4e5dafe6422d94facdf7; no voters:
I20260806 22:26:38.830400 2071 leader_election.cc:288] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:26:38.830510 2082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:38.831144 2082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [term 1 LEADER]: Becoming Leader. State: Replica: 85677bf0a2ca4e5dafe6422d94facdf7, State: Running, Role: LEADER
I20260806 22:26:38.831753 2082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } }
I20260806 22:26:38.832674 2071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:26:38.834645 2083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } } }
I20260806 22:26:38.835006 2083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:38.834754 2084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85677bf0a2ca4e5dafe6422d94facdf7. Latest consensus state: current_term: 1 leader_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85677bf0a2ca4e5dafe6422d94facdf7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34751 } } }
I20260806 22:26:38.835129 2084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:38.837256 2089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:26:38.842693 2089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:26:38.851086 2089 catalog_manager.cc:1347] Generated new cluster ID: 4c804fc803b94e468cd1566eebef5e29
I20260806 22:26:38.851204 2089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:26:38.861864 2089 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:26:38.863766 2089 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:26:38.887360 2089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85677bf0a2ca4e5dafe6422d94facdf7: Generated new TSK 0
I20260806 22:26:38.888548 2089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:26:38.946118 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:39.279410 2117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:39.279703 2117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:39.279806 2117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:39.287057 2117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:39.287160 2117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:39.287276 2117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:26:39.296774 2117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:39.298002 2117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:39.299309 2117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:39.312893 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:39.313045 2127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:39.312836 2129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:39.313848 2117 server_base.cc:1034] running on GCE node
I20260806 22:26:39.314543 2117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:39.315505 2117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:39.316716 2117 hybrid_clock.cc:648] HybridClock initialized: now 1786055199316676 us; error 28 us; skew 500 ppm
I20260806 22:26:39.317041 2117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:39 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:39.320346 2117 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:26:39.321828 2117 webserver.cc:460] Webserver started at http://127.1.8.65:41661/ using document root <none> and password file <none>
I20260806 22:26:39.322543 2117 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:39.322708 2117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:39.323042 2117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:39.326174 2117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "274afa1c65bb46eab7c545cc93fb173a"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "4f66f23a39ec21e1f820abf121a5cfcb"
server_key_iv: "84e2704dd0df3fb18655fc467803524e"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.326974 2117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "274afa1c65bb46eab7c545cc93fb173a"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "4f66f23a39ec21e1f820abf121a5cfcb"
server_key_iv: "84e2704dd0df3fb18655fc467803524e"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.332700 2117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:26:39.336441 2139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.337838 2117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:26:39.338025 2117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "274afa1c65bb46eab7c545cc93fb173a"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "4f66f23a39ec21e1f820abf121a5cfcb"
server_key_iv: "84e2704dd0df3fb18655fc467803524e"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.338297 2117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:39.358067 2117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:39.362171 2117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:39.362658 2117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:39.363675 2117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:39.365082 2117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:39.365156 2117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.365316 2117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:39.365386 2117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.401427 2117 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:39915
I20260806 22:26:39.401465 2348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:39915 every 8 connection(s)
I20260806 22:26:39.403069 2117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:26:39.408149 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2117
I20260806 22:26:39.408394 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:26:39.409226 1057 external_mini_cluster.cc:1428] Setting key 654cd81013c60bcbd20a81db0b8fe5e1
Aug 06 22:26:39 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:39.430367 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:39.405903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:34267 (local address 127.1.8.126:34751)
0806 22:26:39.406364 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:39.406368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:39.407639 (+ 1271us) server_negotiation.cc:407] Connection header received
0806 22:26:39.410546 (+ 2907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:39.410553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:39.410645 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:39.410779 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:39.412955 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.413709 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.417079 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.417482 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.417593 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:39.422759 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:39.422767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:39.422775 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:39.422823 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:39.426715 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:39.427492 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:39.427501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:39.427507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:39.427680 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:39.428257 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:39.428265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:39.428271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:39.428658 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:39.428834 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:39.429562 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 22:26:39.429849 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":346,"thread_start_us":234,"threads_started":1}
I20260806 22:26:39.431069 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:39.406473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34751 (local address 127.1.8.65:34267)
0806 22:26:39.407290 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:39.407330 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:39.410165 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:39.411019 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:39.411027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:39.411843 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:39.412738 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:39.412768 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.413911 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.413920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:39.416871 (+ 2951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:39.416912 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.417769 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.417790 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:39.418246 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:39.419099 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:39.419165 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:39.422451 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:39.426869 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:39.426877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:39.426891 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:39.427335 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:39.427807 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:39.427810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:39.427812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:39.428026 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:39.428793 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:39.428799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:39.429145 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 22:26:39.429381 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 22:26:39.429858 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":331,"threads_started":1}
I20260806 22:26:39.433142 2349 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34751
I20260806 22:26:39.433533 2349 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:39.434527 2349 heartbeater.cc:511] Master 127.1.8.126:34751 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
I20260806 22:26:39.437000 2019 ts_manager.cc:194] Registered new tserver with Master: 274afa1c65bb46eab7c545cc93fb173a (127.1.8.65:39915)
I20260806 22:26:39.438862 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:34267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:26:39.467902 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:39.806478 2358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:39.806795 2358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:39.806980 2358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:39.814529 2358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:39.814709 2358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:39.814896 2358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:26:39.824689 2358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:39.825953 2358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:39.827307 2358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:39.840476 2367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:39.840662 2369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:39.840713 2371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:39.841569 2358 server_base.cc:1034] running on GCE node
I20260806 22:26:39.842175 2358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:39.843070 2358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:39.844304 2358 hybrid_clock.cc:648] HybridClock initialized: now 1786055199844244 us; error 38 us; skew 500 ppm
I20260806 22:26:39.844664 2358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:39 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:39.847796 2358 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:26:39.849447 2358 webserver.cc:460] Webserver started at http://127.1.8.66:33701/ using document root <none> and password file <none>
I20260806 22:26:39.850106 2358 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:39.850214 2358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:39.850477 2358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:39.853690 2358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "c3eede43597543e3ad3574d5b1e90213"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "8d483bdebc716b118bffc4fba458ea62"
server_key_iv: "8aabee2714d2ad81d0dd5ece51ba1b6f"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.854450 2358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "c3eede43597543e3ad3574d5b1e90213"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "8d483bdebc716b118bffc4fba458ea62"
server_key_iv: "8aabee2714d2ad81d0dd5ece51ba1b6f"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.859536 2358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:26:39.862977 2379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.864478 2358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:39.864615 2358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "c3eede43597543e3ad3574d5b1e90213"
format_stamp: "Formatted at 2026-08-06 22:26:39 on dist-test-slave-7vjv"
server_key: "8d483bdebc716b118bffc4fba458ea62"
server_key_iv: "8aabee2714d2ad81d0dd5ece51ba1b6f"
server_key_version: "encryptionkey@0"
I20260806 22:26:39.864771 2358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:39.877462 2358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:39.881424 2358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:39.881861 2358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:39.882929 2358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:39.884361 2358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:39.884469 2358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.884636 2358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:39.884718 2358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:39.920518 2358 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39367
I20260806 22:26:39.920617 2569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39367 every 8 connection(s)
I20260806 22:26:39.921875 2358 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:26:39.930207 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2358
I20260806 22:26:39.930413 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:26:39.930959 1057 external_mini_cluster.cc:1428] Setting key a76211f4965b413ba1d5eed18e72c048
Aug 06 22:26:39 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:39.944397 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:39.925324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:59273 (local address 127.1.8.126:34751)
0806 22:26:39.925554 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:39.925562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:39.926392 (+ 830us) server_negotiation.cc:407] Connection header received
0806 22:26:39.928316 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:39.928320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:39.928390 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:39.928482 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:39.930001 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.931152 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.933034 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.933624 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.933754 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:39.937408 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:39.937417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:39.937424 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:39.937468 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:39.940900 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:39.941945 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:39.941951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:39.941958 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:39.942090 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:39.942602 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:39.942611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:39.942613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:39.942872 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:39.943068 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:39.943808 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 22:26:39.944037 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:26:39.945405 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:39.925360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34751 (local address 127.1.8.66:59273)
0806 22:26:39.926222 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:39.926259 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:39.927887 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:39.928690 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:39.928698 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:39.929231 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:39.929810 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:39.929827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.931299 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.931303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:39.932876 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:39.932895 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:39.933858 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:39.933863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:39.934131 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:39.934866 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:39.934889 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:39.937158 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:39.941056 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:39.941066 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:39.941085 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:39.941782 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:39.942207 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:39.942210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:39.942212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:39.942457 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:39.943017 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:39.943024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:39.943405 (+ 381us) client_negotiation.cc:769] Sending connection context
0806 22:26:39.943705 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 22:26:39.944267 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":350,"threads_started":1}
I20260806 22:26:39.946895 2571 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34751
I20260806 22:26:39.947229 2571 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:39.948144 2571 heartbeater.cc:511] Master 127.1.8.126:34751 requested a full tablet report, sending...
I20260806 22:26:39.949932 2019 ts_manager.cc:194] Registered new tserver with Master: c3eede43597543e3ad3574d5b1e90213 (127.1.8.66:39367)
I20260806 22:26:39.951288 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:59273
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:26:39.981173 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:40.313688 2577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:40.313954 2577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:40.314111 2577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:40.321549 2577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:40.321647 2577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:40.321864 2577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:26:40.331822 2577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:40.333209 2577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:40.334640 2577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:40.356328 2589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:40.356295 2592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:40.356391 2587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:40.357386 2577 server_base.cc:1034] running on GCE node
I20260806 22:26:40.358362 2577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:40.359333 2577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:40.360601 2577 hybrid_clock.cc:648] HybridClock initialized: now 1786055200360531 us; error 61 us; skew 500 ppm
I20260806 22:26:40.360921 2577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:40 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055200, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:40.363987 2577 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:26:40.365485 2577 webserver.cc:460] Webserver started at http://127.1.8.67:32825/ using document root <none> and password file <none>
I20260806 22:26:40.366247 2577 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:40.366431 2577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:40.366763 2577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:40.370031 2577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "e524872a5eb54da3bcac9317ac417aa7"
format_stamp: "Formatted at 2026-08-06 22:26:40 on dist-test-slave-7vjv"
server_key: "4f7992e04e9e20c2a3b70a649e2e6b76"
server_key_iv: "adbfbef1ebc5d373668dc58ef3ba880c"
server_key_version: "encryptionkey@0"
I20260806 22:26:40.370998 2577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "e524872a5eb54da3bcac9317ac417aa7"
format_stamp: "Formatted at 2026-08-06 22:26:40 on dist-test-slave-7vjv"
server_key: "4f7992e04e9e20c2a3b70a649e2e6b76"
server_key_iv: "adbfbef1ebc5d373668dc58ef3ba880c"
server_key_version: "encryptionkey@0"
I20260806 22:26:40.376683 2577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:26:40.380244 2603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:40.381611 2577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:26:40.381786 2577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "e524872a5eb54da3bcac9317ac417aa7"
format_stamp: "Formatted at 2026-08-06 22:26:40 on dist-test-slave-7vjv"
server_key: "4f7992e04e9e20c2a3b70a649e2e6b76"
server_key_iv: "adbfbef1ebc5d373668dc58ef3ba880c"
server_key_version: "encryptionkey@0"
I20260806 22:26:40.381983 2577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:40.400728 2577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:40.405274 2577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:40.405601 2577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:40.406561 2577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:26:40.407881 2577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:40.407955 2577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:40.408180 2577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:40.408248 2577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:40.442232 2349 heartbeater.cc:503] Master 127.1.8.126:34751 was elected leader, sending a full tablet report...
I20260806 22:26:40.448351 2577 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:38419
I20260806 22:26:40.448397 2784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:38419 every 8 connection(s)
I20260806 22:26:40.449707 2577 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:26:40.452538 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2577
I20260806 22:26:40.452719 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:26:40.453251 1057 external_mini_cluster.cc:1428] Setting key 6553b8ca64b40ae8899d204eb404415c
Aug 06 22:26:40 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055200, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:40.470985 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.452360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:44607 (local address 127.1.8.126:34751)
0806 22:26:40.452833 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.452843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.454610 (+ 1767us) server_negotiation.cc:407] Connection header received
0806 22:26:40.456558 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.456562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.456645 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.456918 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:40.458376 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.459136 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.460839 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.461246 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.461383 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.465049 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:40.465054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:40.465057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:40.465087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:40.467983 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.468975 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.468979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.468981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.469103 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.469506 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.469510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.469512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.469774 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:40.469887 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.470541 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.470712 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":242,"threads_started":1}
I20260806 22:26:40.471930 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.453031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34751 (local address 127.1.8.67:44607)
0806 22:26:40.454165 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.454223 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.456307 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.457150 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.457158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.457662 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:40.458208 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.458225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.459300 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.459305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.460681 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.460693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.461497 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.461505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.461754 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.462347 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:40.462373 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:40.464869 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:40.468136 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:40.468145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:40.468160 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:40.468788 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:40.469206 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:40.469209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:40.469211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:40.469401 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:40.469883 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:40.469888 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:40.470202 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.470426 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.470807 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":339,"threads_started":1}
I20260806 22:26:40.473675 2785 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34751
I20260806 22:26:40.474009 2785 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:40.474951 2785 heartbeater.cc:511] Master 127.1.8.126:34751 requested a full tablet report, sending...
I20260806 22:26:40.476856 2019 ts_manager.cc:194] Registered new tserver with Master: e524872a5eb54da3bcac9317ac417aa7 (127.1.8.67:38419)
I20260806 22:26:40.478179 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:44607
I20260806 22:26:40.481437 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:26:40.505972 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.489835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.1.8.126:34751)
0806 22:26:40.490027 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.490031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.490867 (+ 836us) server_negotiation.cc:407] Connection header received
0806 22:26:40.491101 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.491106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.491200 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.491316 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:40.493693 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.494426 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.495788 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.496308 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.496462 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.499400 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:40.499411 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:40.499426 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:40.499459 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:40.502840 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.503565 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.503571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.503574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.503773 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.504245 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.504250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.504252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.504605 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:40.504923 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.505310 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.505601 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:26:40.513515 2019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:40.521576 2019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:40.553009 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.536676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.1.8.67:38419)
0806 22:26:40.541009 (+ 4333us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.541019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.541059 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:26:40.541168 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.541176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.541532 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.543680 (+ 2148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:40.544116 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.546083 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.549771 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.551099 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.551281 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.551698 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.552076 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.552324 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4146,"thread_start_us":3977,"threads_started":1}
I20260806 22:26:40.553088 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.536228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:38419 (local address 127.0.0.1:57400)
0806 22:26:40.537707 (+ 1479us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.537754 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.538009 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.541991 (+ 3982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.542001 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.542033 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:40.542802 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.542826 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.546248 (+ 3422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.546254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.549535 (+ 3281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.549550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.551391 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.551400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.551705 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.551861 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.552029 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.552274 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"mutex_wait_us":60,"thread_start_us":4824,"threads_started":1}
I20260806 22:26:40.553886 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.536547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:39915 (local address 127.0.0.1:38318)
0806 22:26:40.541374 (+ 4827us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.541394 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.541892 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.543305 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.543310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.543336 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:40.544239 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.544254 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.546820 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.546826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.550027 (+ 3201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.550047 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.552739 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.552757 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.553037 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.553078 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.553156 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.553288 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4690,"mutex_wait_us":4,"thread_start_us":532,"threads_started":1}
I20260806 22:26:40.554934 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.537152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.1.8.65:39915)
0806 22:26:40.540318 (+ 3166us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.540331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.541676 (+ 1345us) server_negotiation.cc:407] Connection header received
0806 22:26:40.542097 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.542107 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.542362 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.542777 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:40.544505 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.546658 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.550245 (+ 3587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.552061 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.552239 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.552515 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.554183 (+ 1668us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.554375 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1499,"thread_start_us":1259,"threads_started":1}
I20260806 22:26:40.555796 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.539096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39367 (local address 127.0.0.1:48960)
0806 22:26:40.543493 (+ 4397us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.543512 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.543830 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.545162 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.545168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.545192 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:40.545776 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.545792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.548026 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.548033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.551354 (+ 3321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.551374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.554842 (+ 3468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.554850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.555065 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.555104 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.555214 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.555362 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4235,"mutex_wait_us":3,"thread_start_us":4051,"threads_started":1}
I20260806 22:26:40.556767 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.539899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.1.8.66:39367)
0806 22:26:40.543485 (+ 3586us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.543494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.543894 (+ 400us) server_negotiation.cc:407] Connection header received
0806 22:26:40.544042 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.544059 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.544334 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.544758 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:40.546071 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.547845 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.551620 (+ 3775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.554540 (+ 2920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.554702 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.554947 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.556035 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.556312 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2251,"thread_start_us":2135,"threads_started":1}
I20260806 22:26:40.558092 2677 tablet_service.cc:1467] Processing CreateTablet for tablet 627351335e524cba88e9c206062d2b94 (DEFAULT_TABLE table=test-table [id=6056142a9d864193b0cad6d621ce6988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:40.558122 2226 tablet_service.cc:1467] Processing CreateTablet for tablet 627351335e524cba88e9c206062d2b94 (DEFAULT_TABLE table=test-table [id=6056142a9d864193b0cad6d621ce6988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:40.559898 2677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627351335e524cba88e9c206062d2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:40.560490 2226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627351335e524cba88e9c206062d2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:40.563117 2457 tablet_service.cc:1467] Processing CreateTablet for tablet 627351335e524cba88e9c206062d2b94 (DEFAULT_TABLE table=test-table [id=6056142a9d864193b0cad6d621ce6988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:40.564945 2457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627351335e524cba88e9c206062d2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:40.574676 2813 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Bootstrap starting.
I20260806 22:26:40.580070 2813 tablet_bootstrap.cc:654] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:40.580137 2816 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Bootstrap starting.
I20260806 22:26:40.582536 2813 log.cc:824] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:40.583009 2817 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Bootstrap starting.
I20260806 22:26:40.586408 2816 tablet_bootstrap.cc:654] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:40.587970 2817 tablet_bootstrap.cc:654] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:40.587961 2813 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: No bootstrap required, opened a new log
I20260806 22:26:40.588415 2813 ts_tablet_manager.cc:1397] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 22:26:40.588768 2816 log.cc:824] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:40.589540 2817 log.cc:824] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:40.593222 2816 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: No bootstrap required, opened a new log
I20260806 22:26:40.593531 2816 ts_tablet_manager.cc:1397] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 22:26:40.593942 2817 tablet_bootstrap.cc:492] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: No bootstrap required, opened a new log
I20260806 22:26:40.594319 2817 ts_tablet_manager.cc:1397] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 22:26:40.599401 2813 raft_consensus.cc:348] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.600018 2813 raft_consensus.cc:374] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:40.600210 2813 raft_consensus.cc:729] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e524872a5eb54da3bcac9317ac417aa7, State: Initialized, Role: FOLLOWER
I20260806 22:26:40.601158 2813 consensus_queue.cc:260] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.603021 2785 heartbeater.cc:503] Master 127.1.8.126:34751 was elected leader, sending a full tablet report...
I20260806 22:26:40.603982 2813 ts_tablet_manager.cc:1428] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260806 22:26:40.608402 2816 raft_consensus.cc:348] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.608914 2816 raft_consensus.cc:374] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:40.608711 2817 raft_consensus.cc:348] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.609020 2816 raft_consensus.cc:729] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274afa1c65bb46eab7c545cc93fb173a, State: Initialized, Role: FOLLOWER
I20260806 22:26:40.609161 2817 raft_consensus.cc:374] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:40.609256 2817 raft_consensus.cc:729] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eede43597543e3ad3574d5b1e90213, State: Initialized, Role: FOLLOWER
I20260806 22:26:40.609928 2817 consensus_queue.cc:260] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.609944 2816 consensus_queue.cc:260] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.612030 2571 heartbeater.cc:503] Master 127.1.8.126:34751 was elected leader, sending a full tablet report...
I20260806 22:26:40.612782 2816 ts_tablet_manager.cc:1428] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Time spent starting tablet: real 0.019s user 0.017s sys 0.003s
I20260806 22:26:40.613086 2817 ts_tablet_manager.cc:1428] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
W20260806 22:26:40.656044 2352 tablet.cc:2369] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:26:40.674284 2572 tablet.cc:2369] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:26:40.701704 2786 tablet.cc:2369] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:26:40.860829 2821 raft_consensus.cc:482] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:40.861192 2821 raft_consensus.cc:504] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.863251 2821 leader_election.cc:288] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 274afa1c65bb46eab7c545cc93fb173a (127.1.8.65:39915), c3eede43597543e3ad3574d5b1e90213 (127.1.8.66:39367)
I20260806 22:26:40.870106 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.863768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:57461 (local address 127.1.8.65:39915)
0806 22:26:40.864042 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.864049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.864085 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:26:40.864192 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.864199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.864287 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.864405 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:40.864933 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.865958 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.867941 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.869294 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.869419 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.869491 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.869822 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.869916 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:26:40.870193 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.863523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:39915 (local address 127.1.8.67:57461)
0806 22:26:40.863734 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.863755 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.863908 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.864426 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.864433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.864464 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:40.864759 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.864778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.866212 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.866216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.867786 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.867798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.869440 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.869445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.869638 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.869674 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.869792 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.869928 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:26:40.871287 2266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627351335e524cba88e9c206062d2b94" candidate_uuid: "e524872a5eb54da3bcac9317ac417aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274afa1c65bb46eab7c545cc93fb173a" is_pre_election: true
I20260806 22:26:40.871693 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.864238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:51017 (local address 127.1.8.66:39367)
0806 22:26:40.864434 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.864440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.865142 (+ 702us) server_negotiation.cc:407] Connection header received
0806 22:26:40.865352 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.865358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.865441 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.865527 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:40.866491 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.867186 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.869412 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.870777 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.870883 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.870946 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.871333 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.871422 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:26:40.871806 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.864142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39367 (local address 127.1.8.67:51017)
0806 22:26:40.865020 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:40.865036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:40.865184 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:40.865624 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:40.865629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:40.865654 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:40.866299 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.866313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.867358 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.867362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.869229 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:40.869245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.870977 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.870984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:40.871166 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.871201 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:26:40.871306 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:26:40.871435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":1,"thread_start_us":539,"threads_started":1}
I20260806 22:26:40.871899 2266 raft_consensus.cc:2454] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e524872a5eb54da3bcac9317ac417aa7 in term 0.
I20260806 22:26:40.872745 2606 leader_election.cc:302] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274afa1c65bb46eab7c545cc93fb173a, e524872a5eb54da3bcac9317ac417aa7; no voters:
I20260806 22:26:40.873126 2821 raft_consensus.cc:2781] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:40.873293 2821 raft_consensus.cc:482] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:40.873040 2491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627351335e524cba88e9c206062d2b94" candidate_uuid: "e524872a5eb54da3bcac9317ac417aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eede43597543e3ad3574d5b1e90213" is_pre_election: true
I20260806 22:26:40.873404 2821 raft_consensus.cc:3037] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:40.873683 2491 raft_consensus.cc:2454] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e524872a5eb54da3bcac9317ac417aa7 in term 0.
I20260806 22:26:40.875847 2821 raft_consensus.cc:504] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.876462 2821 leader_election.cc:288] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [CANDIDATE]: Term 1 election: Requested vote from peers 274afa1c65bb46eab7c545cc93fb173a (127.1.8.65:39915), c3eede43597543e3ad3574d5b1e90213 (127.1.8.66:39367)
I20260806 22:26:40.876909 2266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627351335e524cba88e9c206062d2b94" candidate_uuid: "e524872a5eb54da3bcac9317ac417aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274afa1c65bb46eab7c545cc93fb173a"
I20260806 22:26:40.876958 2491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627351335e524cba88e9c206062d2b94" candidate_uuid: "e524872a5eb54da3bcac9317ac417aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eede43597543e3ad3574d5b1e90213"
I20260806 22:26:40.877125 2266 raft_consensus.cc:3037] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:40.877202 2491 raft_consensus.cc:3037] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:40.879374 2266 raft_consensus.cc:2454] T 627351335e524cba88e9c206062d2b94 P 274afa1c65bb46eab7c545cc93fb173a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e524872a5eb54da3bcac9317ac417aa7 in term 1.
I20260806 22:26:40.879513 2491 raft_consensus.cc:2454] T 627351335e524cba88e9c206062d2b94 P c3eede43597543e3ad3574d5b1e90213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e524872a5eb54da3bcac9317ac417aa7 in term 1.
I20260806 22:26:40.879781 2606 leader_election.cc:302] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274afa1c65bb46eab7c545cc93fb173a, e524872a5eb54da3bcac9317ac417aa7; no voters:
I20260806 22:26:40.880074 2821 raft_consensus.cc:2781] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:40.880512 2821 raft_consensus.cc:686] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [term 1 LEADER]: Becoming Leader. State: Replica: e524872a5eb54da3bcac9317ac417aa7, State: Running, Role: LEADER
I20260806 22:26:40.881024 2821 consensus_queue.cc:237] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } }
I20260806 22:26:40.885753 2015 catalog_manager.cc:5495] T 627351335e524cba88e9c206062d2b94 P e524872a5eb54da3bcac9317ac417aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e524872a5eb54da3bcac9317ac417aa7 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "e524872a5eb54da3bcac9317ac417aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e524872a5eb54da3bcac9317ac417aa7" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "274afa1c65bb46eab7c545cc93fb173a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3eede43597543e3ad3574d5b1e90213" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39367 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:26:40 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:40.922749 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.906251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.1.8.65:39915)
0806 22:26:40.906431 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.906436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.907120 (+ 684us) server_negotiation.cc:407] Connection header received
0806 22:26:40.907324 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.907334 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.907416 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.907538 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:40.909063 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.909808 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.911101 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.911514 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.911652 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.915751 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:40.915766 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:40.915784 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:40.915819 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:40.919313 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.920262 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.920270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.920272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.920419 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.920898 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.920902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.920904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.921271 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:40.921539 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.921884 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.922069 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 22:26:40 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:26:40 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:40.990639 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:40.974446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.1.8.65:39915)
0806 22:26:40.974696 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:40.974703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:40.975365 (+ 662us) server_negotiation.cc:407] Connection header received
0806 22:26:40.975598 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:40.975602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:40.975668 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:40.975782 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:40.977216 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.977979 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.979764 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:40.980242 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:40.980380 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:40.984611 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:40.984616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:40.984619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:40.984652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:40.987756 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.988478 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.988485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.988493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.988631 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:40.989046 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:40.989051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:40.989054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:40.989385 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:40.989900 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:40.990034 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:26:40.990251 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:41.047361 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:41.031866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.1.8.65:39915)
0806 22:26:41.032152 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:41.032159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:41.032444 (+ 285us) server_negotiation.cc:407] Connection header received
0806 22:26:41.032669 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:41.032673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:41.032737 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:41.032850 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:41.034160 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.034859 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.036175 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.036600 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.036742 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:41.041449 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:41.041454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:41.041457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:41.041488 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:41.044767 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.045440 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.045443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.045445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.045613 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.046012 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.046015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.046017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.046333 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:41.046589 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:41.046772 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:26:41.046983 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 22:26:41.049351 2191 tablet_service.cc:2908] Rejecting scan request for tablet 627351335e524cba88e9c206062d2b94: Uninitialized: clean time has not yet been initialized
I20260806 22:26:41.052601 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2117
I20260806 22:26:41.065987 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:41.053548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.1.8.65:39915)
0806 22:26:41.053723 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:41.053728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:41.054236 (+ 508us) server_negotiation.cc:407] Connection header received
0806 22:26:41.054346 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:41.054349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:41.054412 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:41.054486 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:41.055688 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.056533 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.057915 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.058661 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.058801 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:41.061492 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:41.061496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:41.061499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:41.061536 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:41.063848 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.064474 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.064478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.064480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.064594 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.064904 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.064908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.064911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.065171 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:41.065380 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:41.065558 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:26:41.065712 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:26:41.066761 2342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:41.159176 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2117
I20260806 22:26:41.185846 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2358
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:26:41.200407 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:41.186764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.1.8.66:39367)
0806 22:26:41.186974 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:41.186979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:41.186998 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:26:41.187140 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:41.187144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:41.187211 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:41.187297 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:41.188596 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.189336 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.190573 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.190954 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.191041 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:41.194990 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:41.195003 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:41.195019 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:41.195062 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:41.197898 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.198535 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.198542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.198544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.198684 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.199015 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.199018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.199020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.199394 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:41.199626 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:41.199868 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:26:41.200022 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:26:41.201220 2562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:41.290459 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2358
I20260806 22:26:41.315297 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2577
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:26:41.333395 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:41.316138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.1.8.67:38419)
0806 22:26:41.316645 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:41.316650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:41.316666 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:26:41.316722 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:41.316726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:41.316790 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:41.316879 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:41.318060 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.318709 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.320145 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:41.320563 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:41.320660 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:41.327628 (+ 6968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:41.327642 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:41.327658 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:41.327690 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:41.330391 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.331033 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.331040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.331042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.331252 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:41.331787 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:41.331790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:41.331792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:41.332234 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:41.332533 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:41.332709 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 22:26:41.332957 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":294,"threads_started":1}
I20260806 22:26:41.334190 2780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:41.424214 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2577
I20260806 22:26:41.450721 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1966
I20260806 22:26:41.452085 2063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:41.561379 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1966
2026-08-06T22:26:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4783 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[2859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[2859](info): set up 2 sockets
Aug 06 22:26:41 dist-test-slave-7vjv krb5kdc[2859](info): commencing operation
krb5kdc: starting...
W20260806 22:26:42.943913 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 22:26:43.114003 1057 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.170s user 0.004s sys 0.004s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 22:26:43.216848 1057 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.103s user 0.000s sys 0.007s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 22:26:43.350510 1057 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.001s sys 0.007s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/test-user.keytab.
W20260806 22:26:43.802253 1057 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.451s user 0.000s sys 0.008s
Aug 06 22:26:43 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:26:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:26:43.897662 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:41097
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:41097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:44.237417 2876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:44.237802 2876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:44.237957 2876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:44.245574 2876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:26:44.245692 2876 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:26:44.245786 2876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:44.245909 2876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:44.246011 2876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:26:44.246124 2876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:26:44.256448 2876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:41097
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:44.257834 2876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:44.259359 2876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:44.272483 2886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:44.272619 2887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:44.272519 2890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:44.273676 2876 server_base.cc:1034] running on GCE node
I20260806 22:26:44.274588 2876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:44.276024 2876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:44.277334 2876 hybrid_clock.cc:648] HybridClock initialized: now 1786055204277245 us; error 56 us; skew 500 ppm
I20260806 22:26:44.277704 2876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:44 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055204, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:44.280989 2876 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:26:44.282449 2876 webserver.cc:460] Webserver started at http://127.1.8.126:46571/ using document root <none> and password file <none>
I20260806 22:26:44.283097 2876 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:44.283182 2876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:44.283452 2876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:44.286510 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "fb774ec74bae4891b9186bece40bed01"
format_stamp: "Formatted at 2026-08-06 22:26:44 on dist-test-slave-7vjv"
server_key: "d75da54ef2ceff3ff7f55f3347abc530"
server_key_iv: "6720b08c8182a14b9ec9691fa8db5753"
server_key_version: "encryptionkey@0"
I20260806 22:26:44.287308 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "fb774ec74bae4891b9186bece40bed01"
format_stamp: "Formatted at 2026-08-06 22:26:44 on dist-test-slave-7vjv"
server_key: "d75da54ef2ceff3ff7f55f3347abc530"
server_key_iv: "6720b08c8182a14b9ec9691fa8db5753"
server_key_version: "encryptionkey@0"
I20260806 22:26:44.292294 2876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:26:44.295456 2898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:44.296830 2876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:44.296958 2876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "fb774ec74bae4891b9186bece40bed01"
format_stamp: "Formatted at 2026-08-06 22:26:44 on dist-test-slave-7vjv"
server_key: "d75da54ef2ceff3ff7f55f3347abc530"
server_key_iv: "6720b08c8182a14b9ec9691fa8db5753"
server_key_version: "encryptionkey@0"
I20260806 22:26:44.297129 2876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:44.338889 2876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:44.344230 2876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:44.344539 2876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:44.366268 2876 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:41097
I20260806 22:26:44.366289 2993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:41097 every 8 connection(s)
I20260806 22:26:44.367702 2876 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:26:44.368132 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2876
I20260806 22:26:44.368368 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:26:44.368983 1057 external_mini_cluster.cc:1428] Setting key fd778f64d8e4d515dddf75196d81ef1a
I20260806 22:26:44.374912 2994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:44.392464 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Bootstrap starting.
I20260806 22:26:44.396396 2994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:44.397488 2994 log.cc:824] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:44.400764 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: No bootstrap required, opened a new log
Aug 06 22:26:44 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:44.411206 2994 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } }
I20260806 22:26:44.411646 2994 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:44.411755 2994 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb774ec74bae4891b9186bece40bed01, State: Initialized, Role: FOLLOWER
I20260806 22:26:44.412024 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:44.371206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.1.8.126:41097)
0806 22:26:44.388605 (+ 17399us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:44.388623 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:44.388680 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:26:44.391206 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:44.391234 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:44.391750 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:44.392259 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:44.393953 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:44.396035 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:44.397365 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:44.398443 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:44.398592 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:44.403109 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:44.403119 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:44.403138 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:44.403181 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:44.407209 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:44.407978 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:44.407986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:44.407994 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:44.408246 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:44.408630 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:44.408636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:44.408642 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:44.409408 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:44.409716 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:44.410236 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 22:26:44.410659 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17087,"thread_start_us":16785,"threads_started":1}
I20260806 22:26:44.412451 2994 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } }
I20260806 22:26:44.412750 2994 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:26:44.412896 2994 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:26:44.413081 2994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:44.415591 2994 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } }
I20260806 22:26:44.416024 2994 leader_election.cc:302] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb774ec74bae4891b9186bece40bed01; no voters:
I20260806 22:26:44.416663 2994 leader_election.cc:288] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:26:44.416842 3000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:44.418093 3000 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [term 1 LEADER]: Becoming Leader. State: Replica: fb774ec74bae4891b9186bece40bed01, State: Running, Role: LEADER
I20260806 22:26:44.418678 3000 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } }
I20260806 22:26:44.418883 2994 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:26:44.421657 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb774ec74bae4891b9186bece40bed01. Latest consensus state: current_term: 1 leader_uuid: "fb774ec74bae4891b9186bece40bed01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } } }
I20260806 22:26:44.422183 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:44.422073 3001 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb774ec74bae4891b9186bece40bed01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb774ec74bae4891b9186bece40bed01" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41097 } } }
I20260806 22:26:44.422379 3001 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01 [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:44.423009 3011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:26:44.428135 3011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:26:44.438597 3011 catalog_manager.cc:1347] Generated new cluster ID: d88309f3881f48da88ede2dc7b35579d
I20260806 22:26:44.438881 3011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:26:44.454039 3011 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:26:44.455322 3011 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:26:44.471410 3011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Generated new TSK 0
I20260806 22:26:44.472473 3011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:26:44.543385 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:41097
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:45.462179 3034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:45.463029 3034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:45.463485 3034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:45.478816 3034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:45.479317 3034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:45.479703 3034 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:45.480069 3034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:26:45.500806 3034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:45.504374 3034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:45.508311 3034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:45.542995 3044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:45.543370 3042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:45.543480 3049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:45.549635 3034 server_base.cc:1034] running on GCE node
I20260806 22:26:45.551364 3034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:45.555047 3034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:45.558771 3034 hybrid_clock.cc:648] HybridClock initialized: now 1786055205558461 us; error 260 us; skew 500 ppm
I20260806 22:26:45.559255 3034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:45 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:45.568640 3034 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:26:45.574723 3034 webserver.cc:460] Webserver started at http://127.1.8.65:39161/ using document root <none> and password file <none>
I20260806 22:26:45.577584 3034 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:45.577888 3034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:45.579300 3034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:45.590584 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "97abb639187d463ca537a6331d16b202"
format_stamp: "Formatted at 2026-08-06 22:26:45 on dist-test-slave-7vjv"
server_key: "3c8884544711790c8df76afedf57fade"
server_key_iv: "935fdc1f463f447ec29db094ccf80032"
server_key_version: "encryptionkey@0"
I20260806 22:26:45.593895 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "97abb639187d463ca537a6331d16b202"
format_stamp: "Formatted at 2026-08-06 22:26:45 on dist-test-slave-7vjv"
server_key: "3c8884544711790c8df76afedf57fade"
server_key_iv: "935fdc1f463f447ec29db094ccf80032"
server_key_version: "encryptionkey@0"
I20260806 22:26:45.617465 3034 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.022s sys 0.000s
I20260806 22:26:45.631464 3063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:45.635416 3034 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.000s
I20260806 22:26:45.635746 3034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "97abb639187d463ca537a6331d16b202"
format_stamp: "Formatted at 2026-08-06 22:26:45 on dist-test-slave-7vjv"
server_key: "3c8884544711790c8df76afedf57fade"
server_key_iv: "935fdc1f463f447ec29db094ccf80032"
server_key_version: "encryptionkey@0"
I20260806 22:26:45.636255 3034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:45.655911 3034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:45.660068 3034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:45.660380 3034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:45.663340 3034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:45.663475 3034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:45.663661 3034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:45.663755 3034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:45 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:45.687151 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:45.663844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:43695 (local address 127.1.8.126:41097)
0806 22:26:45.664937 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:45.664945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:45.665794 (+ 849us) server_negotiation.cc:407] Connection header received
0806 22:26:45.667887 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:45.667897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:45.668010 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:45.668193 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:45.670048 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.671586 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.674023 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.674590 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.674746 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:45.679622 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:45.679631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:45.679634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:45.679689 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:45.682925 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:45.683972 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:45.683981 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:45.683985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:45.684222 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:45.684849 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:45.684854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:45.684857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:45.685305 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:45.685498 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:45.686594 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0806 22:26:45.686795 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":786,"threads_started":1}
I20260806 22:26:45.689182 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:45.664468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.65:43695)
0806 22:26:45.665593 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:45.665641 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:45.667578 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:45.668388 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:45.668400 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:45.668948 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:45.669843 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:45.669881 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.671816 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.671827 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:45.673865 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:45.673881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.674872 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.674882 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:45.675210 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:45.676046 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:45.676068 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:45.679273 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:45.683099 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:45.683110 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:45.683128 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:45.683798 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:45.684348 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:45.684353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:45.684359 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:45.684721 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:45.685506 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:45.685518 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:45.686075 (+ 557us) client_negotiation.cc:769] Sending connection context
0806 22:26:45.686506 (+ 431us) client_negotiation.cc:240] Negotiation successful
0806 22:26:45.687203 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":505,"threads_started":1}
I20260806 22:26:45.713239 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:45.696169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:46129 (local address 127.1.8.126:41097)
0806 22:26:45.696380 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:45.696385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:45.697176 (+ 791us) server_negotiation.cc:407] Connection header received
0806 22:26:45.697395 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:45.697399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:45.697458 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:45.697528 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:45.699038 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.699816 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.702589 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.703097 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.703191 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:45.706425 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:45.706435 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:45.706441 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:45.706485 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:45.710146 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:45.711208 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:45.711216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:45.711224 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:45.711342 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:45.711733 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:45.711744 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:45.711746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:45.712021 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:45.712203 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:45.712674 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 22:26:45.712913 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:26:45.713390 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:45.696010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.65:46129)
0806 22:26:45.697063 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:45.697091 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:45.697260 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:45.697594 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:45.697602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:45.698347 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:45.698875 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:45.698891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.699956 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.699964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:45.702365 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:45.702382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:45.703286 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:45.703297 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:45.703566 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:45.704436 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:45.704474 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:45.706201 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:45.710358 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:45.710367 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:45.710374 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:45.711076 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:45.711460 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:45.711465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:45.711469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:45.711609 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:45.712189 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:45.712194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:45.712408 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 22:26:45.712553 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 22:26:45.712946 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":688,"threads_started":1}
I20260806 22:26:45.726619 3034 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:38911
I20260806 22:26:45.726660 3276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:38911 every 8 connection(s)
I20260806 22:26:45.727861 3034 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:26:45.730084 3277 heartbeater.cc:347] Connected to a master server at 127.1.8.126:41097
I20260806 22:26:45.730326 3277 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:45.730973 3277 heartbeater.cc:511] Master 127.1.8.126:41097 requested a full tablet report, sending...
I20260806 22:26:45.731196 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3034
I20260806 22:26:45.731482 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:26:45.732364 1057 external_mini_cluster.cc:1428] Setting key 16a2ae7e6d3b5326a7dd40d4f57dd0f4
I20260806 22:26:45.733206 2933 ts_manager.cc:194] Registered new tserver with Master: 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911)
I20260806 22:26:45.735162 2933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:43695
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:26:45.781710 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:41097
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:46.138950 3282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:46.139232 3282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:46.139343 3282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:46.146898 3282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:46.147020 3282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:46.147148 3282 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:46.147300 3282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:26:46.156831 3282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:46.158072 3282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:46.159405 3282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:46.173311 3292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:46.173255 3293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:46.173292 3296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:46.174513 3282 server_base.cc:1034] running on GCE node
I20260806 22:26:46.175135 3282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:46.176075 3282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:46.177376 3282 hybrid_clock.cc:648] HybridClock initialized: now 1786055206177300 us; error 43 us; skew 500 ppm
I20260806 22:26:46.177700 3282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:46.181007 3282 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:26:46.182585 3282 webserver.cc:460] Webserver started at http://127.1.8.66:32957/ using document root <none> and password file <none>
I20260806 22:26:46.183292 3282 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:46.183414 3282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:46.183776 3282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:46.186872 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "02524779870041bea6057a97c1f33f63"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "5f2353739a9d5a4289bead9be6c20b36"
server_key_iv: "bcb0d6aed24d387b67a82673a7a86c78"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.187716 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "02524779870041bea6057a97c1f33f63"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "5f2353739a9d5a4289bead9be6c20b36"
server_key_iv: "bcb0d6aed24d387b67a82673a7a86c78"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.193149 3282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 22:26:46.196630 3305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:46.197934 3282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:26:46.198086 3282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "02524779870041bea6057a97c1f33f63"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "5f2353739a9d5a4289bead9be6c20b36"
server_key_iv: "bcb0d6aed24d387b67a82673a7a86c78"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.198299 3282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:46.212796 3282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:46.217440 3282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:46.217738 3282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:46.220808 3282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:46.220898 3282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:46.221060 3282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:46.221120 3282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:46.251237 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.221524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:45975 (local address 127.1.8.126:41097)
0806 22:26:46.224852 (+ 3328us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.224862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.225608 (+ 746us) server_negotiation.cc:407] Connection header received
0806 22:26:46.228511 (+ 2903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.228518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.228613 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.228738 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:46.231184 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.232439 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.235372 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.236269 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.236446 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.241612 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:46.241631 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:46.241641 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:46.241678 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:46.246081 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.247127 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.247131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.247133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.247251 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.247837 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.247845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.247849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.248265 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:46.248450 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:46.250589 (+ 2139us) server_negotiation.cc:299] Negotiation successful
0806 22:26:46.250857 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3136,"thread_start_us":2986,"threads_started":1}
I20260806 22:26:46.251546 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.222101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.66:45975)
0806 22:26:46.225411 (+ 3310us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.225462 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.228066 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.229001 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.229017 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.229798 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:46.230903 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.230957 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.232614 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.232620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.235171 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.235183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.236572 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.236587 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.236935 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.237753 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:46.237796 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:46.241344 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:46.246259 (+ 4915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.246271 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.246296 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.246960 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.247384 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.247389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.247392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.247624 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.248466 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:46.248475 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:46.248890 (+ 415us) client_negotiation.cc:769] Sending connection context
0806 22:26:46.249297 (+ 407us) client_negotiation.cc:240] Negotiation successful
0806 22:26:46.249881 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2991,"thread_start_us":2768,"threads_started":1}
I20260806 22:26:46.275045 3282 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:36845
I20260806 22:26:46.275100 3494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:36845 every 8 connection(s)
I20260806 22:26:46.276746 3282 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:26:46.278470 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.260332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:57333 (local address 127.1.8.126:41097)
0806 22:26:46.260540 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.260546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.261573 (+ 1027us) server_negotiation.cc:407] Connection header received
0806 22:26:46.261882 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.261895 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.261987 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.262121 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:46.264409 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.265167 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.267726 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.268234 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.268380 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.271300 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:46.271308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:46.271314 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:46.271361 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:46.275434 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.276132 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.276138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.276142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.276310 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.276675 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.276683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.276687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.277128 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:46.277564 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:46.277856 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 22:26:46.278095 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:26:46.278590 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.260393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.66:57333)
0806 22:26:46.261457 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.261483 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.261716 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.262180 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.262189 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.263469 (+ 1280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:46.264218 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.264238 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.265309 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.265317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.267537 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.267566 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.268487 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.268496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.268657 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.269447 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:46.269480 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:46.271049 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:46.275568 (+ 4519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.275572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.275574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.275970 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.276431 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.276434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.276436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.276563 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.277299 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:46.277307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:46.277596 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 22:26:46.277757 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 22:26:46.278137 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":689,"threads_started":1}
I20260806 22:26:46.279269 3497 heartbeater.cc:347] Connected to a master server at 127.1.8.126:41097
I20260806 22:26:46.279779 3497 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:46.280758 3497 heartbeater.cc:511] Master 127.1.8.126:41097 requested a full tablet report, sending...
I20260806 22:26:46.282373 2933 ts_manager.cc:194] Registered new tserver with Master: 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845)
I20260806 22:26:46.283378 2933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:45975
I20260806 22:26:46.283968 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3282
I20260806 22:26:46.284230 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:26:46.284668 1057 external_mini_cluster.cc:1428] Setting key 75097959b0b77068a39487b1cce8211c
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:26:46.337248 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:41097
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:46.723026 3502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:46.723346 3502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:46.723539 3502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:46.731604 3502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:46.731755 3502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:46.731935 3502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:46.732084 3502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:26:46.737890 3277 heartbeater.cc:503] Master 127.1.8.126:41097 was elected leader, sending a full tablet report...
I20260806 22:26:46.743573 3502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:46.744987 3502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:46.746395 3502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:46.760676 3510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:46.760720 3514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:46.760751 3511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:46.761336 3502 server_base.cc:1034] running on GCE node
I20260806 22:26:46.762089 3502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:46.763085 3502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:46.764289 3502 hybrid_clock.cc:648] HybridClock initialized: now 1786055206764204 us; error 57 us; skew 500 ppm
I20260806 22:26:46.764559 3502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:46.767668 3502 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:26:46.769136 3502 webserver.cc:460] Webserver started at http://127.1.8.67:41127/ using document root <none> and password file <none>
I20260806 22:26:46.769769 3502 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:46.769856 3502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:46.770131 3502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:46.773128 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "ac9d49fb43a14284a2780b8917096447"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "a69085873a75d907dc067be6ebfa92fe"
server_key_iv: "7d115cef7ff0ee49d5f72b46a497ebee"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.773905 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "ac9d49fb43a14284a2780b8917096447"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "a69085873a75d907dc067be6ebfa92fe"
server_key_iv: "7d115cef7ff0ee49d5f72b46a497ebee"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.779256 3502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 22:26:46.782866 3521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:46.784265 3502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:26:46.784413 3502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "ac9d49fb43a14284a2780b8917096447"
format_stamp: "Formatted at 2026-08-06 22:26:46 on dist-test-slave-7vjv"
server_key: "a69085873a75d907dc067be6ebfa92fe"
server_key_iv: "7d115cef7ff0ee49d5f72b46a497ebee"
server_key_version: "encryptionkey@0"
I20260806 22:26:46.784605 3502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:46.798704 3502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:46.803857 3502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:46.804203 3502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:46.807020 3502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:46.807117 3502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:46.807341 3502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:46.807442 3502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:46.834388 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.808737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.67:46755)
0806 22:26:46.809811 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.809858 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.812241 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.813268 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.813279 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.814137 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:46.815097 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.815117 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.817097 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.817105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.819324 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.819345 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.820613 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.820618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.820962 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.821898 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:46.821941 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:46.825523 (+ 3582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:46.829739 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.829749 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.829770 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.830311 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.830800 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.830806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.830811 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.831113 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.831829 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:46.831844 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:46.832295 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 22:26:46.832567 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 22:26:46.833045 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":502,"threads_started":1}
I20260806 22:26:46.836169 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.807954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:46755 (local address 127.1.8.126:41097)
0806 22:26:46.808669 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.808675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.810034 (+ 1359us) server_negotiation.cc:407] Connection header received
0806 22:26:46.812644 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.812654 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.812768 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.812958 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:46.815324 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.816897 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.819494 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.820352 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.820478 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.825839 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:46.825848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:46.825858 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:46.825892 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:46.829574 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.830457 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.830463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.830466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.830615 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.831238 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.831245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.831248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.831645 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:46.835361 (+ 3716us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:46.835502 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:26:46.835698 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":394,"threads_started":1}
I20260806 22:26:46.863971 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.845312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:42473 (local address 127.1.8.126:41097)
0806 22:26:46.845501 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.845509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.848911 (+ 3402us) server_negotiation.cc:407] Connection header received
0806 22:26:46.849114 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.849118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.849185 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.849304 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:46.851272 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.852304 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.853975 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.854476 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.854625 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.856951 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:46.856961 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:46.856968 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:46.857012 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:46.860540 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.861402 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.861409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.861413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.861576 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.862139 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.862147 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.862153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.862604 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:46.862787 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:46.863328 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 22:26:46.863571 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:26:46.864212 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.844950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.1.8.67:42473)
0806 22:26:46.848722 (+ 3772us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.848747 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.848989 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.849330 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.849335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.850219 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:46.851028 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.851062 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.852463 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.852468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.853776 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.853789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.854681 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.854687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:46.854872 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.855541 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:46.855559 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:46.856653 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:46.860677 (+ 4024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.860683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.860687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.861250 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.861698 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:46.861703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:46.861707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:46.861977 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:46.862771 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:46.862777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:46.863017 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 22:26:46.863208 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 22:26:46.863590 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3562,"thread_start_us":3335,"threads_started":1}
I20260806 22:26:46.866250 3502 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:37859
I20260806 22:26:46.866282 3726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:37859 every 8 connection(s)
I20260806 22:26:46.867900 3502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:26:46.870223 3727 heartbeater.cc:347] Connected to a master server at 127.1.8.126:41097
I20260806 22:26:46.870471 3727 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:46.871199 3727 heartbeater.cc:511] Master 127.1.8.126:41097 requested a full tablet report, sending...
I20260806 22:26:46.872637 2933 ts_manager.cc:194] Registered new tserver with Master: ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859)
I20260806 22:26:46.873499 2933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:46755
I20260806 22:26:46.877769 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3502
I20260806 22:26:46.878001 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:26:46.878443 1057 external_mini_cluster.cc:1428] Setting key 8cbaafad105ff32df62c51ccc1d0b8d4
I20260806 22:26:46.881757 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:26:46 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:46.963267 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.945018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.1.8.126:41097)
0806 22:26:46.945242 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.945249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.945496 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:26:46.945748 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.945757 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.945822 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.945929 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:46.947820 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.949463 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.951002 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.951449 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:46.951597 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:46.957116 (+ 5519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:46.957121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:46.957124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:46.957156 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:46.960270 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.960981 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.960986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.960990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.961177 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:46.961595 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:46.961601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:46.961604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:46.961893 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:46.962350 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:46.962499 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:26:46.962766 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:26:46.968003 2933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:46.977195 2933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:47.006991 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.992585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.1.8.66:36845)
0806 22:26:46.994293 (+ 1708us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.994302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.994334 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:26:46.994461 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.994466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.994638 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.995233 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:46.997793 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:46.999975 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.004085 (+ 4110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.005740 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.005873 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.006118 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.006487 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.006631 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1540,"thread_start_us":1250,"threads_started":1}
I20260806 22:26:47.007236 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.992003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:36845 (local address 127.0.0.1:58672)
0806 22:26:46.993077 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.993118 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.993431 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.995074 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.995091 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.995132 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:46.997363 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.997381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.000168 (+ 2787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.000174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.003853 (+ 3679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.003862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.005953 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.005959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.006172 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.006243 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.006420 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.006635 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":151,"thread_start_us":827,"threads_started":1}
I20260806 22:26:47.007616 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.996126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44454 (local address 127.1.8.65:38911)
0806 22:26:46.997686 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.997696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.998619 (+ 923us) server_negotiation.cc:407] Connection header received
0806 22:26:46.998821 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.998827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.998988 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.999202 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:47.000751 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.001889 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.004626 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.005774 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.005940 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.006176 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.006852 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.007093 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":1183,"threads_started":1}
I20260806 22:26:47.007747 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.995056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38911 (local address 127.0.0.1:44454)
0806 22:26:46.998476 (+ 3420us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.998490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.998659 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.999271 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.999276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.999301 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:47.000088 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.000126 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.002076 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.002080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.004417 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.004434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.006425 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.006433 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.006643 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.006673 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.006757 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.006892 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3332,"mutex_wait_us":32,"thread_start_us":3104,"threads_started":1}
I20260806 22:26:47.008874 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.992805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.1.8.67:37859)
0806 22:26:46.994317 (+ 1512us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:46.994325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:46.995825 (+ 1500us) server_negotiation.cc:407] Connection header received
0806 22:26:46.995930 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:46.995938 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:46.996245 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:46.996512 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:46.998466 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.000318 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.004083 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.005740 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.005873 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.006132 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.008236 (+ 2104us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.008406 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1317,"thread_start_us":1044,"threads_started":1}
I20260806 22:26:47.008905 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:46.993548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37859 (local address 127.0.0.1:34900)
0806 22:26:46.995464 (+ 1916us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:46.995484 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:46.995697 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:46.996960 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:46.996965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:46.996986 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:46.998225 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:46.998243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.000452 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.000465 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.003816 (+ 3351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.003847 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.007551 (+ 3704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.007559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.007806 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.007847 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.007938 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.008063 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"mutex_wait_us":91,"thread_start_us":1518,"threads_started":1}
I20260806 22:26:47.011451 3381 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 (DEFAULT_TABLE table=test-table [id=bfaccd795c604c808413753f9ff8b6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.011894 3615 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 (DEFAULT_TABLE table=test-table [id=bfaccd795c604c808413753f9ff8b6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.012732 3381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ec6c5278f64bffbf0d026049b6b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.013383 3615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ec6c5278f64bffbf0d026049b6b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.014830 3157 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 (DEFAULT_TABLE table=test-table [id=bfaccd795c604c808413753f9ff8b6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.016292 3157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ec6c5278f64bffbf0d026049b6b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.026011 3756 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Bootstrap starting.
I20260806 22:26:47.029422 3757 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Bootstrap starting.
I20260806 22:26:47.029898 3756 tablet_bootstrap.cc:654] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.032291 3756 log.cc:824] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:47.033888 3757 tablet_bootstrap.cc:654] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.034511 3760 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Bootstrap starting.
I20260806 22:26:47.035679 3757 log.cc:824] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:47.036348 3756 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: No bootstrap required, opened a new log
I20260806 22:26:47.036796 3756 ts_tablet_manager.cc:1397] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260806 22:26:47.038798 3760 tablet_bootstrap.cc:654] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.039423 3757 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: No bootstrap required, opened a new log
I20260806 22:26:47.039850 3757 ts_tablet_manager.cc:1397] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 22:26:47.040481 3760 log.cc:824] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:47.043875 3760 tablet_bootstrap.cc:492] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: No bootstrap required, opened a new log
I20260806 22:26:47.044212 3760 ts_tablet_manager.cc:1397] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260806 22:26:47.051301 3756 raft_consensus.cc:348] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.051909 3756 raft_consensus.cc:374] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.052062 3756 raft_consensus.cc:729] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02524779870041bea6057a97c1f33f63, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.052977 3756 consensus_queue.cc:260] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.053915 3760 raft_consensus.cc:348] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.054059 3757 raft_consensus.cc:348] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.054503 3760 raft_consensus.cc:374] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.054539 3757 raft_consensus.cc:374] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.054627 3760 raft_consensus.cc:729] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97abb639187d463ca537a6331d16b202, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.054687 3757 raft_consensus.cc:729] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9d49fb43a14284a2780b8917096447, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.055291 3757 consensus_queue.cc:260] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.055485 3497 heartbeater.cc:503] Master 127.1.8.126:41097 was elected leader, sending a full tablet report...
I20260806 22:26:47.056419 3760 consensus_queue.cc:260] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.060031 3756 ts_tablet_manager.cc:1428] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Time spent starting tablet: real 0.023s user 0.012s sys 0.005s
I20260806 22:26:47.060161 3727 heartbeater.cc:503] Master 127.1.8.126:41097 was elected leader, sending a full tablet report...
I20260806 22:26:47.062163 3757 ts_tablet_manager.cc:1428] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Time spent starting tablet: real 0.022s user 0.009s sys 0.013s
I20260806 22:26:47.063918 3760 ts_tablet_manager.cc:1428] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Time spent starting tablet: real 0.020s user 0.005s sys 0.012s
W20260806 22:26:47.119927 3728 tablet.cc:2369] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:26:47.230770 3278 tablet.cc:2369] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:26:47.260689 3769 raft_consensus.cc:482] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:47.261245 3769 raft_consensus.cc:504] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.263542 3769 leader_election.cc:288] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859), 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911)
I20260806 22:26:47.271230 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.264153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:35683 (local address 127.1.8.65:38911)
0806 22:26:47.264402 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.264410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.265632 (+ 1222us) server_negotiation.cc:407] Connection header received
0806 22:26:47.265870 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.265875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.265956 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.266071 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:47.266722 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.267392 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.269420 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.270190 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.270337 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.270483 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.270807 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.270948 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:26:47.271267 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.264123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:58533 (local address 127.1.8.67:37859)
0806 22:26:47.264394 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.264404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.265462 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 22:26:47.265632 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.265640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.265742 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.265838 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:47.266576 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.267344 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.269387 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.270208 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.270366 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.270491 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.270888 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.271033 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:26:47.271427 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.263746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37859 (local address 127.1.8.66:58533)
0806 22:26:47.265327 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:47.265346 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:47.265487 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:47.265872 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:47.265877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:47.265901 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:47.266417 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.266433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.267477 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.267485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.269220 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.269232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.270545 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.270552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.270689 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.270712 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.270830 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.270950 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1461,"mutex_wait_us":23,"thread_start_us":1014,"threads_started":1}
I20260806 22:26:47.271812 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.264207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38911 (local address 127.1.8.66:35683)
0806 22:26:47.265532 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:47.265552 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:47.265691 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:47.266084 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:47.266087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:47.266105 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:47.266562 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.266575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.267556 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.267563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.269243 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.269255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.270490 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.270498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.270647 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.270671 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.270747 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.270878 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"mutex_wait_us":3,"thread_start_us":761,"threads_started":1}
I20260806 22:26:47.272696 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ec6c5278f64bffbf0d026049b6b3e4" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447" is_pre_election: true
I20260806 22:26:47.272994 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ec6c5278f64bffbf0d026049b6b3e4" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202" is_pre_election: true
I20260806 22:26:47.273408 3643 raft_consensus.cc:2454] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 0.
I20260806 22:26:47.273588 3196 raft_consensus.cc:2454] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 0.
I20260806 22:26:47.274271 3310 leader_election.cc:302] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02524779870041bea6057a97c1f33f63, ac9d49fb43a14284a2780b8917096447; no voters:
I20260806 22:26:47.274762 3769 raft_consensus.cc:2781] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:47.274940 3769 raft_consensus.cc:482] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:47.275096 3769 raft_consensus.cc:3037] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:47.278779 3769 raft_consensus.cc:504] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.279589 3769 leader_election.cc:288] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 election: Requested vote from peers ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859), 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911)
I20260806 22:26:47.280148 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ec6c5278f64bffbf0d026049b6b3e4" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202"
I20260806 22:26:47.280184 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ec6c5278f64bffbf0d026049b6b3e4" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447"
I20260806 22:26:47.280386 3643 raft_consensus.cc:3037] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:47.280376 3196 raft_consensus.cc:3037] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:26:47.280740 3498 tablet.cc:2369] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:26:47.283509 3196 raft_consensus.cc:2454] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 1.
I20260806 22:26:47.283715 3643 raft_consensus.cc:2454] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 1.
I20260806 22:26:47.283926 3308 leader_election.cc:302] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02524779870041bea6057a97c1f33f63, 97abb639187d463ca537a6331d16b202; no voters:
I20260806 22:26:47.284348 3769 raft_consensus.cc:2781] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:47.284828 3769 raft_consensus.cc:686] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 1 LEADER]: Becoming Leader. State: Replica: 02524779870041bea6057a97c1f33f63, State: Running, Role: LEADER
I20260806 22:26:47.285408 3769 consensus_queue.cc:237] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } }
I20260806 22:26:47.291205 2933 catalog_manager.cc:5495] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02524779870041bea6057a97c1f33f63 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } health_report { overall_health: UNKNOWN } } }
I20260806 22:26:47.385639 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.379524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.1.8.66:36845)
0806 22:26:47.379764 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.379771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.379960 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:26:47.380353 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.380357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.380437 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.380594 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:47.380984 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.381740 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.383303 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.383757 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.383915 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.384538 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:47.385035 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:47.385189 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.385293 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.385373 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:26:47.394434 3196 raft_consensus.cc:1261] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Refusing update from remote peer 02524779870041bea6057a97c1f33f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:47.394454 3643 raft_consensus.cc:1261] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Refusing update from remote peer 02524779870041bea6057a97c1f33f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:47.395486 3782 consensus_queue.cc:1035] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:47.395953 3769 consensus_queue.cc:1035] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:47.407306 3789 mvcc.cc:205] Tried to move back new op lower bound from 7315682129476825088 to 7315682129045229568. Current Snapshot: MvccSnapshot[applied={T|T < 7315682129476825088}]
I20260806 22:26:47.412029 3793 mvcc.cc:205] Tried to move back new op lower bound from 7315682129476825088 to 7315682129045229568. Current Snapshot: MvccSnapshot[applied={T|T < 7315682129476825088}]
I20260806 22:26:47.428810 2933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:47.429178 2933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:47.433578 2933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Sending DeleteTablet for 3 replicas of tablet 62ec6c5278f64bffbf0d026049b6b3e4
I20260806 22:26:47.434976 3381 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:47 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:58672
I20260806 22:26:47.434957 3615 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:47 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:34900
I20260806 22:26:47.435670 3157 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec6c5278f64bffbf0d026049b6b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:47 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:44454
I20260806 22:26:47.439841 2933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:47.441500 2933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:47.451927 3814 tablet_replica.cc:330] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: stopping tablet replica
I20260806 22:26:47.452502 3814 raft_consensus.cc:2229] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:47.452927 3614 tablet_service.cc:1467] Processing CreateTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 (DEFAULT_TABLE table=test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.453125 3814 raft_consensus.cc:2258] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:47.453158 3380 tablet_service.cc:1467] Processing CreateTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 (DEFAULT_TABLE table=test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.453609 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cab922a0fab47728ad31ebf91ecc987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.453975 3380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cab922a0fab47728ad31ebf91ecc987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.457785 3814 ts_tablet_manager.cc:1905] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:47.462683 3157 tablet_service.cc:1467] Processing CreateTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 (DEFAULT_TABLE table=test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:47.463265 3157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cab922a0fab47728ad31ebf91ecc987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.468957 3812 tablet_replica.cc:330] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: stopping tablet replica
I20260806 22:26:47.469487 3812 raft_consensus.cc:2229] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:47.470194 3812 raft_consensus.cc:2258] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:47.477317 3757 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Bootstrap starting.
I20260806 22:26:47.477403 3760 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Bootstrap starting.
I20260806 22:26:47.478009 3813 tablet_replica.cc:330] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: stopping tablet replica
I20260806 22:26:47.478669 3813 raft_consensus.cc:2229] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:47.479792 3757 tablet_bootstrap.cc:654] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.479795 3760 tablet_bootstrap.cc:654] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.480000 3813 raft_consensus.cc:2258] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:47.480475 3756 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Bootstrap starting.
I20260806 22:26:47.483703 3760 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: No bootstrap required, opened a new log
I20260806 22:26:47.483843 3812 ts_tablet_manager.cc:1905] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:47.483954 3760 ts_tablet_manager.cc:1397] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 22:26:47.485376 3760 raft_consensus.cc:348] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.485664 3760 raft_consensus.cc:374] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.485778 3760 raft_consensus.cc:729] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97abb639187d463ca537a6331d16b202, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.486085 3756 tablet_bootstrap.cc:654] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.486197 3760 consensus_queue.cc:260] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.486963 3760 ts_tablet_manager.cc:1428] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:26:47.488710 3813 ts_tablet_manager.cc:1905] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:47.499109 3814 ts_tablet_manager.cc:1918] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:26:47.499513 3814 log.cc:1196] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/62ec6c5278f64bffbf0d026049b6b3e4
I20260806 22:26:47.500639 3814 ts_tablet_manager.cc:1939] T 62ec6c5278f64bffbf0d026049b6b3e4 P 97abb639187d463ca537a6331d16b202: Deleting consensus metadata
I20260806 22:26:47.500669 3812 ts_tablet_manager.cc:1918] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:26:47.500962 3812 log.cc:1196] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/62ec6c5278f64bffbf0d026049b6b3e4
I20260806 22:26:47.501878 3812 ts_tablet_manager.cc:1939] T 62ec6c5278f64bffbf0d026049b6b3e4 P ac9d49fb43a14284a2780b8917096447: Deleting consensus metadata
I20260806 22:26:47.503484 2902 catalog_manager.cc:4841] TS 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911): tablet 62ec6c5278f64bffbf0d026049b6b3e4 (table test-table [id=bfaccd795c604c808413753f9ff8b6b1]) successfully deleted
I20260806 22:26:47.504501 2903 catalog_manager.cc:4841] TS ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859): tablet 62ec6c5278f64bffbf0d026049b6b3e4 (table test-table [id=bfaccd795c604c808413753f9ff8b6b1]) successfully deleted
I20260806 22:26:47.507417 3757 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: No bootstrap required, opened a new log
I20260806 22:26:47.507843 3757 ts_tablet_manager.cc:1397] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.011s
I20260806 22:26:47.509331 3757 raft_consensus.cc:348] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.509636 3757 raft_consensus.cc:374] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.509742 3757 raft_consensus.cc:729] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9d49fb43a14284a2780b8917096447, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.510058 3757 consensus_queue.cc:260] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.510874 3756 tablet_bootstrap.cc:492] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: No bootstrap required, opened a new log
I20260806 22:26:47.511173 3756 ts_tablet_manager.cc:1397] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.009s
I20260806 22:26:47.511679 3813 ts_tablet_manager.cc:1918] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:26:47.512058 3813 log.cc:1196] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/62ec6c5278f64bffbf0d026049b6b3e4
I20260806 22:26:47.512234 3757 ts_tablet_manager.cc:1428] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 22:26:47.513697 3813 ts_tablet_manager.cc:1939] T 62ec6c5278f64bffbf0d026049b6b3e4 P 02524779870041bea6057a97c1f33f63: Deleting consensus metadata
I20260806 22:26:47.513868 3756 raft_consensus.cc:348] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.514196 3756 raft_consensus.cc:374] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.514372 3756 raft_consensus.cc:729] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02524779870041bea6057a97c1f33f63, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.514745 3756 consensus_queue.cc:260] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.516215 2902 catalog_manager.cc:4841] TS 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845): tablet 62ec6c5278f64bffbf0d026049b6b3e4 (table test-table [id=bfaccd795c604c808413753f9ff8b6b1]) successfully deleted
I20260806 22:26:47.516362 3756 ts_tablet_manager.cc:1428] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260806 22:26:47.774482 3774 raft_consensus.cc:482] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:47.774786 3774 raft_consensus.cc:504] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.777168 3774 leader_election.cc:288] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845), ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859)
I20260806 22:26:47.799052 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.777417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37859 (local address 127.1.8.65:49077)
0806 22:26:47.789227 (+ 11810us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:47.789246 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:47.789339 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:47.790656 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:47.790663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:47.790706 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:47.791922 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.791940 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.793491 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.793504 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.795624 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.795638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.797718 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.797727 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.798039 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.798075 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.798254 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.798419 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11688,"mutex_wait_us":237,"thread_start_us":11275,"threads_started":1}
I20260806 22:26:47.799201 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.777722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:49077 (local address 127.1.8.67:37859)
0806 22:26:47.790132 (+ 12410us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.790143 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.790186 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:26:47.790276 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.790284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.790436 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.790579 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:47.792218 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.793322 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.795809 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.797445 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.797648 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.797775 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.798451 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.798634 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12176,"thread_start_us":12044,"threads_started":1}
I20260806 22:26:47.799888 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.777748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60857 (local address 127.1.8.66:36845)
0806 22:26:47.778066 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.778075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.788861 (+ 10786us) server_negotiation.cc:407] Connection header received
0806 22:26:47.789363 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.789372 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.789467 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.789586 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:47.790912 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.792262 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.796033 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.797930 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.798163 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.798325 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.798905 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.799047 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:26:47.800586 3819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.777417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:36845 (local address 127.1.8.65:60857)
0806 22:26:47.788718 (+ 11301us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:47.788745 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:47.789153 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:47.789910 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:47.789916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:47.789994 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:47.790599 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.790619 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.792475 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.792481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.795770 (+ 3289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.795797 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.798381 (+ 2584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.798393 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.798630 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.798669 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.799571 (+ 902us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.799724 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11140,"mutex_wait_us":5,"thread_start_us":10994,"threads_started":1}
I20260806 22:26:47.800484 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cab922a0fab47728ad31ebf91ecc987" candidate_uuid: "97abb639187d463ca537a6331d16b202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447" is_pre_election: true
I20260806 22:26:47.800944 3643 raft_consensus.cc:2454] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97abb639187d463ca537a6331d16b202 in term 0.
I20260806 22:26:47.801695 3068 leader_election.cc:302] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97abb639187d463ca537a6331d16b202, ac9d49fb43a14284a2780b8917096447; no voters:
I20260806 22:26:47.802610 3774 raft_consensus.cc:2781] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:47.802776 3774 raft_consensus.cc:482] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:47.802611 3420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cab922a0fab47728ad31ebf91ecc987" candidate_uuid: "97abb639187d463ca537a6331d16b202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02524779870041bea6057a97c1f33f63" is_pre_election: true
I20260806 22:26:47.802899 3774 raft_consensus.cc:3037] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:47.803138 3420 raft_consensus.cc:2454] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97abb639187d463ca537a6331d16b202 in term 0.
I20260806 22:26:47.807188 3774 raft_consensus.cc:504] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.808200 3774 leader_election.cc:288] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [CANDIDATE]: Term 1 election: Requested vote from peers 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845), ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859)
I20260806 22:26:47.808758 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cab922a0fab47728ad31ebf91ecc987" candidate_uuid: "97abb639187d463ca537a6331d16b202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447"
I20260806 22:26:47.808998 3643 raft_consensus.cc:3037] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:47.808933 3420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cab922a0fab47728ad31ebf91ecc987" candidate_uuid: "97abb639187d463ca537a6331d16b202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02524779870041bea6057a97c1f33f63"
I20260806 22:26:47.809185 3420 raft_consensus.cc:3037] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:47.811847 3643 raft_consensus.cc:2454] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97abb639187d463ca537a6331d16b202 in term 1.
I20260806 22:26:47.812390 3068 leader_election.cc:302] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97abb639187d463ca537a6331d16b202, ac9d49fb43a14284a2780b8917096447; no voters:
I20260806 22:26:47.812757 3774 raft_consensus.cc:2781] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:47.813611 3774 raft_consensus.cc:686] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 1 LEADER]: Becoming Leader. State: Replica: 97abb639187d463ca537a6331d16b202, State: Running, Role: LEADER
I20260806 22:26:47.813936 3420 raft_consensus.cc:2454] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97abb639187d463ca537a6331d16b202 in term 1.
I20260806 22:26:47.814059 3774 consensus_queue.cc:237] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.818548 2928 catalog_manager.cc:5495] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97abb639187d463ca537a6331d16b202 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "97abb639187d463ca537a6331d16b202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:26:47 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055204, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:47.915400 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.878405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.1.8.126:41097)
0806 22:26:47.896573 (+ 18168us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:47.896580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:47.897217 (+ 637us) server_negotiation.cc:407] Connection header received
0806 22:26:47.897435 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:47.897442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:47.897538 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:47.897726 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:47.899530 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.901192 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.902940 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.903554 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.903700 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.908578 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:47.908583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:47.908586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:47.908614 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:47.911861 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:47.912994 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:47.912999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:47.913001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:47.913128 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:47.913647 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:47.913651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:47.913653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:47.913904 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:47.914030 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:47.914656 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 22:26:47.915025 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":13505,"thread_start_us":13166,"threads_started":1}
I20260806 22:26:47.915792 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:47.878546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41097 (local address 127.0.0.1:45842)
0806 22:26:47.897061 (+ 18515us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:47.897083 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:47.897252 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:47.897788 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:47.897793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:47.898855 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:47.899317 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.899328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.901330 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.901333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.902806 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:47.902818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:47.903736 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:47.903743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:47.903966 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:47.904840 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:47.904857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:47.908308 (+ 3451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:47.912010 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:47.912035 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:47.912046 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:47.912864 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:47.913258 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:47.913264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:47.913268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:47.913489 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:47.914059 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:47.914070 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:47.914313 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 22:26:47.914561 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 22:26:47.914913 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18310,"thread_start_us":17913,"threads_started":1}
I20260806 22:26:47.922782 2928 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:45842:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:26:47.924520 2928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:47.933136 3157 tablet_service.cc:1467] Processing CreateTablet for tablet 83f1788e888d48f6b9503557a74fbc2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bc7ec8cd21a46989e1e339bfb3c9851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:47.933275 3615 tablet_service.cc:1467] Processing CreateTablet for tablet 83f1788e888d48f6b9503557a74fbc2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bc7ec8cd21a46989e1e339bfb3c9851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:47.933373 3381 tablet_service.cc:1467] Processing CreateTablet for tablet 83f1788e888d48f6b9503557a74fbc2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bc7ec8cd21a46989e1e339bfb3c9851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:47.933799 3615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f1788e888d48f6b9503557a74fbc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.933828 3157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f1788e888d48f6b9503557a74fbc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.934012 3381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f1788e888d48f6b9503557a74fbc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:47.942927 3760 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: Bootstrap starting.
I20260806 22:26:47.944578 3756 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: Bootstrap starting.
I20260806 22:26:47.945365 3760 tablet_bootstrap.cc:654] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.947252 3756 tablet_bootstrap.cc:654] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.947926 3757 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: Bootstrap starting.
I20260806 22:26:47.950615 3757 tablet_bootstrap.cc:654] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:47.951094 3756 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: No bootstrap required, opened a new log
I20260806 22:26:47.951364 3756 ts_tablet_manager.cc:1397] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 22:26:47.953003 3760 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: No bootstrap required, opened a new log
I20260806 22:26:47.952970 3756 raft_consensus.cc:348] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.953277 3760 ts_tablet_manager.cc:1397] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 22:26:47.953457 3756 raft_consensus.cc:374] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.953622 3756 raft_consensus.cc:729] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02524779870041bea6057a97c1f33f63, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.954080 3760 raft_consensus.cc:348] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.954043 3756 consensus_queue.cc:260] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.954336 3760 raft_consensus.cc:374] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.954455 3760 raft_consensus.cc:729] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97abb639187d463ca537a6331d16b202, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.954990 3756 ts_tablet_manager.cc:1428] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 22:26:47.954794 3760 consensus_queue.cc:260] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.954840 3769 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:47.955420 3769 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:47.955746 3760 ts_tablet_manager.cc:1428] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 22:26:47.955628 3774 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:47.956938 3757 tablet_bootstrap.cc:492] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: No bootstrap required, opened a new log
I20260806 22:26:47.957177 3757 ts_tablet_manager.cc:1397] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 22:26:47.958218 3757 raft_consensus.cc:348] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.958621 3757 raft_consensus.cc:374] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:47.958806 3757 raft_consensus.cc:729] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9d49fb43a14284a2780b8917096447, State: Initialized, Role: FOLLOWER
I20260806 22:26:47.959143 3757 consensus_queue.cc:260] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:47.960048 3774 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:47.960974 3757 ts_tablet_manager.cc:1428] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:47.959725 3773 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:47.961269 3773 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:48.132983 3773 raft_consensus.cc:482] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:48.133263 3773 raft_consensus.cc:504] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:48.135017 3773 leader_election.cc:288] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911), 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845)
I20260806 22:26:48.140738 3769 raft_consensus.cc:482] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:48.140934 3769 raft_consensus.cc:504] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:48.141656 3769 leader_election.cc:288] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911), ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859)
I20260806 22:26:48.142241 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447" is_pre_election: true
I20260806 22:26:48.142310 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202" is_pre_election: true
I20260806 22:26:48.142522 3643 raft_consensus.cc:2454] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 0.
I20260806 22:26:48.142601 3196 raft_consensus.cc:2454] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 0.
I20260806 22:26:48.142977 3310 leader_election.cc:302] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02524779870041bea6057a97c1f33f63, ac9d49fb43a14284a2780b8917096447; no voters:
I20260806 22:26:48.143344 3769 raft_consensus.cc:2781] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:48.143538 3769 raft_consensus.cc:482] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:48.143715 3769 raft_consensus.cc:3037] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:48.145460 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.136160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:36845 (local address 127.1.8.67:53295)
0806 22:26:48.136997 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:48.137031 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:48.137172 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:48.137588 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:48.137592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:48.137689 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:48.138306 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.138316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.139623 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.139629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.142041 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.142062 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.144312 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.144321 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.144592 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.144619 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 22:26:48.144730 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:26:48.144869 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":45,"thread_start_us":474,"threads_started":1}
I20260806 22:26:48.145460 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.136411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:53295 (local address 127.1.8.66:36845)
0806 22:26:48.136587 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.136594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.137139 (+ 545us) server_negotiation.cc:407] Connection header received
0806 22:26:48.137336 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.137343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.137431 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.137536 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:48.138462 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.139483 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.142238 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.143820 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.143925 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.144110 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.144758 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.144918 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:26:48.146124 3420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "ac9d49fb43a14284a2780b8917096447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02524779870041bea6057a97c1f33f63" is_pre_election: true
I20260806 22:26:48.146277 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.135251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38911 (local address 127.1.8.67:45277)
0806 22:26:48.135804 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:48.135830 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:48.135987 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:48.136856 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:48.136864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:48.136899 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:48.137497 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.137514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.139115 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.139122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.142159 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.142174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.144514 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.144521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.144758 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.144850 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 22:26:48.144943 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:26:48.145116 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":3,"thread_start_us":254,"threads_started":1}
I20260806 22:26:48.146942 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.135363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:45277 (local address 127.1.8.65:38911)
0806 22:26:48.136185 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.136191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.136210 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:26:48.136286 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.136291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.136374 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.136498 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:48.137738 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.138919 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.142363 (+ 3444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.144256 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.144379 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.144507 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.145276 (+ 769us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.145433 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":451,"threads_started":1}
I20260806 22:26:48.147678 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "ac9d49fb43a14284a2780b8917096447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202" is_pre_election: true
I20260806 22:26:48.147801 3769 raft_consensus.cc:504] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:48.147975 3196 raft_consensus.cc:2454] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac9d49fb43a14284a2780b8917096447 in term 0.
I20260806 22:26:48.148376 3420 raft_consensus.cc:2379] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac9d49fb43a14284a2780b8917096447 in current term 1: Already voted for candidate 02524779870041bea6057a97c1f33f63 in this term.
I20260806 22:26:48.148530 3524 leader_election.cc:302] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97abb639187d463ca537a6331d16b202, ac9d49fb43a14284a2780b8917096447; no voters:
I20260806 22:26:48.149024 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202"
I20260806 22:26:48.149091 3773 raft_consensus.cc:2781] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:48.149274 3196 raft_consensus.cc:3037] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:48.149354 3773 raft_consensus.cc:482] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:48.149274 3643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "02524779870041bea6057a97c1f33f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9d49fb43a14284a2780b8917096447"
I20260806 22:26:48.149595 3773 raft_consensus.cc:3037] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:48.149710 3769 leader_election.cc:288] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 election: Requested vote from peers 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911), ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859)
I20260806 22:26:48.152035 3196 raft_consensus.cc:2454] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02524779870041bea6057a97c1f33f63 in term 1.
I20260806 22:26:48.152290 3773 raft_consensus.cc:504] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:48.152442 3308 leader_election.cc:302] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02524779870041bea6057a97c1f33f63, 97abb639187d463ca537a6331d16b202; no voters:
I20260806 22:26:48.152767 3769 raft_consensus.cc:2781] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:48.152803 3643 raft_consensus.cc:2379] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02524779870041bea6057a97c1f33f63 in current term 1: Already voted for candidate ac9d49fb43a14284a2780b8917096447 in this term.
I20260806 22:26:48.153110 3773 leader_election.cc:288] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [CANDIDATE]: Term 1 election: Requested vote from peers 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911), 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845)
I20260806 22:26:48.153753 3420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "ac9d49fb43a14284a2780b8917096447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02524779870041bea6057a97c1f33f63"
I20260806 22:26:48.154091 3769 raft_consensus.cc:686] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [term 1 LEADER]: Becoming Leader. State: Replica: 02524779870041bea6057a97c1f33f63, State: Running, Role: LEADER
I20260806 22:26:48.154455 3196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f1788e888d48f6b9503557a74fbc2c" candidate_uuid: "ac9d49fb43a14284a2780b8917096447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97abb639187d463ca537a6331d16b202"
I20260806 22:26:48.154461 3769 consensus_queue.cc:237] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } }
I20260806 22:26:48.154779 3196 raft_consensus.cc:2379] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ac9d49fb43a14284a2780b8917096447 in current term 1: Already voted for candidate 02524779870041bea6057a97c1f33f63 in this term.
I20260806 22:26:48.156381 3524 leader_election.cc:302] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac9d49fb43a14284a2780b8917096447; no voters: 02524779870041bea6057a97c1f33f63, 97abb639187d463ca537a6331d16b202
I20260806 22:26:48.156719 3773 raft_consensus.cc:2726] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 22:26:48.157167 3841 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: TxnStatusTablet state changed. Reason: New leader 02524779870041bea6057a97c1f33f63. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.157480 3841 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:48.158179 2933 catalog_manager.cc:5495] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02524779870041bea6057a97c1f33f63 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } health_report { overall_health: UNKNOWN } } }
I20260806 22:26:48.159343 3843 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:26:48.159613 3843 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:26:48.202548 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.194738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.1.8.66:36845)
0806 22:26:48.194997 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.195003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.195225 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:26:48.195533 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.195537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.195607 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.195691 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:48.196226 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.197269 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.199738 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.200690 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.200787 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.201638 (+ 851us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.201808 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.201886 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.202074 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.202160 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:26:48.202848 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.194853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:36845 (local address 127.0.0.1:58684)
0806 22:26:48.195113 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:48.195138 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:48.195333 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:48.195759 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:48.195766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:48.195782 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:48.196045 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.196058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.197405 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.197413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.199553 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.199570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.200896 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.200904 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.201258 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.201339 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.201928 (+ 589us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.201954 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:26:48.202047 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:26:48.202248 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:26:48.208559 3196 raft_consensus.cc:1261] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Refusing update from remote peer 02524779870041bea6057a97c1f33f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:48.208701 3643 raft_consensus.cc:1261] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Refusing update from remote peer 02524779870041bea6057a97c1f33f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:48.209424 3841 consensus_queue.cc:1035] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:48.209846 3769 consensus_queue.cc:1035] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:48.212949 3822 consensus_queue.cc:1035] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:48.217167 3774 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: TxnStatusTablet state changed. Reason: New leader 02524779870041bea6057a97c1f33f63. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.217552 3774 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:48.222904 3850 mvcc.cc:205] Tried to move back new op lower bound from 7315682132814254080 to 7315682132604002304. Current Snapshot: MvccSnapshot[applied={T|T < 7315682132814254080}]
I20260806 22:26:48.223553 3773 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: TxnStatusTablet state changed. Reason: New leader 02524779870041bea6057a97c1f33f63. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.224133 3773 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:48.225414 3769 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.225773 3769 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:48.234340 3774 consensus_queue.cc:1035] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:48.246335 3841 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.246837 3841 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 02524779870041bea6057a97c1f33f63: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:48.246898 3822 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.247535 3822 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P 97abb639187d463ca537a6331d16b202: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:48.253247 3773 tablet_replica.cc:439] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02524779870041bea6057a97c1f33f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97abb639187d463ca537a6331d16b202" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38911 } } peers { permanent_uuid: "02524779870041bea6057a97c1f33f63" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 36845 } } peers { permanent_uuid: "ac9d49fb43a14284a2780b8917096447" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37859 } } }
I20260806 22:26:48.253710 3773 tablet_replica.cc:442] T 83f1788e888d48f6b9503557a74fbc2c P ac9d49fb43a14284a2780b8917096447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:48.269239 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.259838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.1.8.65:38911)
0806 22:26:48.260065 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.260071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.261899 (+ 1828us) server_negotiation.cc:407] Connection header received
0806 22:26:48.262189 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.262193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.262333 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.262474 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:48.263212 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.264004 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.266409 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.267032 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.267130 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.267861 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.268629 (+ 768us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.268743 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.268885 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.268956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:26:48.297246 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.284255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:59911 (local address 127.1.8.66:36845)
0806 22:26:48.284471 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.284477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.284980 (+ 503us) server_negotiation.cc:407] Connection header received
0806 22:26:48.285218 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.285224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.285304 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.285394 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:48.286441 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.287923 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.290429 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.291206 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.291339 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.291917 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.292070 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.296342 (+ 4272us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.296654 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.296750 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:26:48.298189 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.283676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:36845 (local address 127.1.8.65:59911)
0806 22:26:48.284791 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:48.284841 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:48.285076 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:48.285436 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:48.285442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:48.285456 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:48.286224 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.286242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.288125 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.288131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.290259 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.290293 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.291412 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.291421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.291638 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.291712 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.296421 (+ 4709us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.296461 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:26:48.297486 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0806 22:26:48.297650 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":662,"threads_started":1}
I20260806 22:26:48.356053 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.345917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:36265 (local address 127.1.8.65:38911)
0806 22:26:48.346164 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.346173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.347828 (+ 1655us) server_negotiation.cc:407] Connection header received
0806 22:26:48.348136 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.348144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.348237 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.348324 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:48.349307 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.350504 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.353276 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.354087 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.354185 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.355117 (+ 932us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.355265 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.355335 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.355593 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.355681 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:26:48.357113 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.344326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38911 (local address 127.1.8.66:36265)
0806 22:26:48.347690 (+ 3364us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:48.347706 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:48.347967 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:48.348515 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:48.348520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:48.348533 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:48.349130 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.349144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.350706 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.350711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.353113 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:48.353129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.354532 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.354544 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:48.354803 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.354903 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:48.355434 (+ 531us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:48.355447 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:26:48.356296 (+ 849us) client_negotiation.cc:240] Negotiation successful
0806 22:26:48.356490 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3114,"thread_start_us":2792,"threads_started":1}
I20260806 22:26:48.416082 2927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:48.416484 2927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:48.420796 2927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb774ec74bae4891b9186bece40bed01: Sending DeleteTablet for 3 replicas of tablet 0cab922a0fab47728ad31ebf91ecc987
I20260806 22:26:48.421886 3615 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:48 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:34900
I20260806 22:26:48.421909 3381 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:48 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:58672
I20260806 22:26:48.422832 3157 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cab922a0fab47728ad31ebf91ecc987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:48 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:44454
I20260806 22:26:48.434583 3883 tablet_replica.cc:330] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: stopping tablet replica
I20260806 22:26:48.434907 3883 raft_consensus.cc:2229] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:48.435312 3883 raft_consensus.cc:2258] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:48.436760 3883 ts_tablet_manager.cc:1905] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:48.446376 3883 ts_tablet_manager.cc:1918] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:48.446584 3883 log.cc:1196] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/0cab922a0fab47728ad31ebf91ecc987
I20260806 22:26:48.447306 3883 ts_tablet_manager.cc:1939] T 0cab922a0fab47728ad31ebf91ecc987 P ac9d49fb43a14284a2780b8917096447: Deleting consensus metadata
I20260806 22:26:48.448853 2903 catalog_manager.cc:4841] TS ac9d49fb43a14284a2780b8917096447 (127.1.8.67:37859): tablet 0cab922a0fab47728ad31ebf91ecc987 (table test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]) successfully deleted
I20260806 22:26:48.451345 3885 tablet_replica.cc:330] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: stopping tablet replica
I20260806 22:26:48.451753 3885 raft_consensus.cc:2229] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:48.452577 3885 raft_consensus.cc:2258] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:48.453784 3884 tablet_replica.cc:330] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: stopping tablet replica
I20260806 22:26:48.454101 3884 raft_consensus.cc:2229] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:48.454521 3884 raft_consensus.cc:2258] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:48.456003 3884 ts_tablet_manager.cc:1905] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:48.461369 3885 ts_tablet_manager.cc:1905] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:48.466866 3884 ts_tablet_manager.cc:1918] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:48.467165 3884 log.cc:1196] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/0cab922a0fab47728ad31ebf91ecc987
I20260806 22:26:48.467859 3884 ts_tablet_manager.cc:1939] T 0cab922a0fab47728ad31ebf91ecc987 P 02524779870041bea6057a97c1f33f63: Deleting consensus metadata
I20260806 22:26:48.469310 2902 catalog_manager.cc:4841] TS 02524779870041bea6057a97c1f33f63 (127.1.8.66:36845): tablet 0cab922a0fab47728ad31ebf91ecc987 (table test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]) successfully deleted
I20260806 22:26:48.470674 3885 ts_tablet_manager.cc:1918] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:48.470866 3885 log.cc:1196] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/0cab922a0fab47728ad31ebf91ecc987
I20260806 22:26:48.471572 3885 ts_tablet_manager.cc:1939] T 0cab922a0fab47728ad31ebf91ecc987 P 97abb639187d463ca537a6331d16b202: Deleting consensus metadata
I20260806 22:26:48.475874 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.438708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.1.8.65:38911)
0806 22:26:48.438955 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.438965 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.449293 (+ 10328us) server_negotiation.cc:407] Connection header received
0806 22:26:48.449587 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.449594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.449717 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.449815 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:48.453366 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.454720 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.457244 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.458189 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.458295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.466006 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:48.466023 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:48.466046 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:48.466095 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:48.471452 (+ 5357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.472629 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.472639 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.472643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.472832 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.473363 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.473369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.473372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.474100 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:48.474506 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.475136 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.475414 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 22:26:48.477264 2902 catalog_manager.cc:4841] TS 97abb639187d463ca537a6331d16b202 (127.1.8.65:38911): tablet 0cab922a0fab47728ad31ebf91ecc987 (table test-table [id=fd8781a26b0b4c2c973453e0cf884e7b]) successfully deleted
W20260806 22:26:48.477761 3271 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44472
I20260806 22:26:48.525580 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.484681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.1.8.126:41097)
0806 22:26:48.500296 (+ 15615us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.500303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.501278 (+ 975us) server_negotiation.cc:407] Connection header received
0806 22:26:48.501556 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.501563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.501661 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.501828 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:48.504483 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.506289 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.508722 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.509570 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.509728 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.514826 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:48.514836 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:48.514841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:48.514918 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:48.520312 (+ 5394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.521499 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.521505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.521510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.521766 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.522293 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.522299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.522303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.522809 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:48.523199 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.524713 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.525030 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15390,"thread_start_us":15620,"threads_started":1}
W20260806 22:26:48.529075 2927 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45854
I20260806 22:26:48.532586 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3034
I20260806 22:26:48.558135 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.536417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.1.8.65:38911)
0806 22:26:48.536623 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.536630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.537205 (+ 575us) server_negotiation.cc:407] Connection header received
0806 22:26:48.537433 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.537443 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.537557 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.537671 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:48.539610 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.541034 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.542936 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.543694 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.543800 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.547798 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:48.547806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:48.547828 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:48.547885 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:48.553363 (+ 5478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.554682 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.554689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.554693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.554928 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.555479 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.555485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.555488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.555891 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:48.557226 (+ 1335us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.557392 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.557644 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 22:26:48.558962 3271 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44482
W20260806 22:26:48.559947 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:48.560286 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3034
I20260806 22:26:48.587368 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3282
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:26:48.605098 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.588303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.1.8.66:36845)
0806 22:26:48.588535 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.588544 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.588614 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:26:48.588880 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.588886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.588996 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.589138 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:48.590690 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.591473 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.593112 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.593618 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.593769 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.598266 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:48.598285 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:48.598301 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:48.598344 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:48.601838 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.602585 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.602593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.602596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.602742 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.603214 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.603218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.603220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.603607 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:48.603994 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.604485 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.604771 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 22:26:48.605967 3488 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58690
W20260806 22:26:48.606666 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3282: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:48.606868 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3282
I20260806 22:26:48.656323 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3502
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[2859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:26:48.672143 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:48.657146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.1.8.67:37859)
0806 22:26:48.657638 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:48.657643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:48.657659 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:26:48.657712 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:48.657716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:48.657793 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:48.657915 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:48.659244 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.659956 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.661278 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:48.661700 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:48.661844 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:48.666204 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:48.666220 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:48.666237 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:48.666275 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:48.669189 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.669933 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.669940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.669943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.670087 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:48.670461 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:48.670467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:48.670472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:48.670845 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:48.671183 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:48.671495 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:26:48.671744 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
W20260806 22:26:48.672890 3718 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34908
W20260806 22:26:48.673588 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3502: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:48.673754 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3502
I20260806 22:26:48.700619 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2876
I20260806 22:26:48.701892 2988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:48.842144 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2876
2026-08-06T22:26:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (7285 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[3909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[3909](info): set up 2 sockets
Aug 06 22:26:48 dist-test-slave-7vjv krb5kdc[3909](info): commencing operation
krb5kdc: starting...
W20260806 22:26:50.221882 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:26:50 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:26:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:26:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:26:50.459771 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38171
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:50.799860 3931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:50.800318 3931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:50.800491 3931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:50.808298 3931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:26:50.808420 3931 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:26:50.808501 3931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:50.808624 3931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:50.808704 3931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:26:50.808820 3931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:26:50.819324 3931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38171
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:50.820806 3931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:50.822256 3931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:50.835253 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:50.835270 3936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:50.835270 3942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:50.836486 3931 server_base.cc:1034] running on GCE node
I20260806 22:26:50.837433 3931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:50.838969 3931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:50.840279 3931 hybrid_clock.cc:648] HybridClock initialized: now 1786055210840180 us; error 45 us; skew 500 ppm
I20260806 22:26:50.840710 3931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:50 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055210, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:50.844355 3931 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:26:50.846105 3931 webserver.cc:460] Webserver started at http://127.1.8.126:39993/ using document root <none> and password file <none>
I20260806 22:26:50.846930 3931 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:50.847070 3931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:50.847457 3931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:50.850950 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "42b9440962d84d819b4595a03f02ba9f"
format_stamp: "Formatted at 2026-08-06 22:26:50 on dist-test-slave-7vjv"
server_key: "b60f4cada688537fb7b1ddcdae01020d"
server_key_iv: "89066ca41cc3eafdeb8b15ce6e4e2a1b"
server_key_version: "encryptionkey@0"
I20260806 22:26:50.851807 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "42b9440962d84d819b4595a03f02ba9f"
format_stamp: "Formatted at 2026-08-06 22:26:50 on dist-test-slave-7vjv"
server_key: "b60f4cada688537fb7b1ddcdae01020d"
server_key_iv: "89066ca41cc3eafdeb8b15ce6e4e2a1b"
server_key_version: "encryptionkey@0"
I20260806 22:26:50.857590 3931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 22:26:50.861394 3954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:50.862859 3931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260806 22:26:50.863066 3931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "42b9440962d84d819b4595a03f02ba9f"
format_stamp: "Formatted at 2026-08-06 22:26:50 on dist-test-slave-7vjv"
server_key: "b60f4cada688537fb7b1ddcdae01020d"
server_key_iv: "89066ca41cc3eafdeb8b15ce6e4e2a1b"
server_key_version: "encryptionkey@0"
I20260806 22:26:50.863300 3931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:50.881434 3931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:50.886014 3931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:50.886369 3931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:50.910058 3931 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38171
I20260806 22:26:50.910069 4047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38171 every 8 connection(s)
I20260806 22:26:50.911530 3931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:26:50.914214 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3931
I20260806 22:26:50.914464 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:26:50.915398 1057 external_mini_cluster.cc:1428] Setting key 9c2566878ca279559d9bf7e7842b2827
I20260806 22:26:50.916365 4048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:50.929386 4048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Bootstrap starting.
Aug 06 22:26:50 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:50.933001 4048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:50.934162 4048 log.cc:824] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:50.937240 4048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: No bootstrap required, opened a new log
I20260806 22:26:50.938684 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:50.917755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51682 (local address 127.1.8.126:38171)
0806 22:26:50.918570 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:50.918582 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:50.918624 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:26:50.920360 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:50.920383 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:50.920803 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:50.921235 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:50.922795 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:50.924308 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:50.925605 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:50.926328 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:50.926566 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:50.930997 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:50.931006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:50.931020 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:50.931075 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:50.934452 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:50.935181 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:50.935188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:50.935194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:50.935373 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:50.935803 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:50.935809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:50.935812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:50.936435 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:50.936741 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:50.937214 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 22:26:50.937660 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":393,"threads_started":1}
I20260806 22:26:50.945818 4048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } }
I20260806 22:26:50.946233 4048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:50.946359 4048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b9440962d84d819b4595a03f02ba9f, State: Initialized, Role: FOLLOWER
I20260806 22:26:50.946928 4048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } }
I20260806 22:26:50.947184 4048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:26:50.947336 4048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:26:50.947522 4048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:50.950034 4048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } }
I20260806 22:26:50.950554 4048 leader_election.cc:302] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42b9440962d84d819b4595a03f02ba9f; no voters:
I20260806 22:26:50.951191 4048 leader_election.cc:288] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:26:50.951330 4056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:50.952126 4056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [term 1 LEADER]: Becoming Leader. State: Replica: 42b9440962d84d819b4595a03f02ba9f, State: Running, Role: LEADER
I20260806 22:26:50.952711 4056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } }
I20260806 22:26:50.953441 4048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:26:50.955341 4057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42b9440962d84d819b4595a03f02ba9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } } }
I20260806 22:26:50.955687 4057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:50.957619 4058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42b9440962d84d819b4595a03f02ba9f. Latest consensus state: current_term: 1 leader_uuid: "42b9440962d84d819b4595a03f02ba9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b9440962d84d819b4595a03f02ba9f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38171 } } }
I20260806 22:26:50.957917 4058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f [sys.catalog]: This master's current role is: LEADER
I20260806 22:26:50.959452 4062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:26:50.965528 4062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:26:50.975010 4062 catalog_manager.cc:1347] Generated new cluster ID: eb4044924029493290acd6c229ad2c42
I20260806 22:26:50.975159 4062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:26:51.020077 4062 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:26:51.021488 4062 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:26:51.027817 4062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Generated new TSK 0
I20260806 22:26:51.028812 4062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:26:51.088310 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38171
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:51.434566 4091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:51.434921 4091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:51.435105 4091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:51.442603 4091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:51.442762 4091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:51.442934 4091 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:26:51.443027 4091 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:51.443146 4091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:26:51.453258 4091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:51.454731 4091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:51.456084 4091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:51.469985 4098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:51.470036 4099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:51.470057 4102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:51.470940 4091 server_base.cc:1034] running on GCE node
I20260806 22:26:51.471594 4091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:51.472644 4091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:51.473882 4091 hybrid_clock.cc:648] HybridClock initialized: now 1786055211473842 us; error 52 us; skew 500 ppm
I20260806 22:26:51.474221 4091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:51 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:51.480777 4091 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:26:51.482383 4091 webserver.cc:460] Webserver started at http://127.1.8.65:40347/ using document root <none> and password file <none>
I20260806 22:26:51.483146 4091 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:51.483304 4091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:51.483647 4091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:51.486928 4091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "655077bd63d94cb6b6351feb9ffdfb37"
format_stamp: "Formatted at 2026-08-06 22:26:51 on dist-test-slave-7vjv"
server_key: "8102301b019a8c1a50acef51446aa02b"
server_key_iv: "a34cda025d1af69bf34ca29b2c7eb700"
server_key_version: "encryptionkey@0"
I20260806 22:26:51.487743 4091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "655077bd63d94cb6b6351feb9ffdfb37"
format_stamp: "Formatted at 2026-08-06 22:26:51 on dist-test-slave-7vjv"
server_key: "8102301b019a8c1a50acef51446aa02b"
server_key_iv: "a34cda025d1af69bf34ca29b2c7eb700"
server_key_version: "encryptionkey@0"
I20260806 22:26:51.493420 4091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:26:51.496941 4117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:51.498294 4091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:26:51.498482 4091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "655077bd63d94cb6b6351feb9ffdfb37"
format_stamp: "Formatted at 2026-08-06 22:26:51 on dist-test-slave-7vjv"
server_key: "8102301b019a8c1a50acef51446aa02b"
server_key_iv: "a34cda025d1af69bf34ca29b2c7eb700"
server_key_version: "encryptionkey@0"
I20260806 22:26:51.498704 4091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:51.512059 4091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:51.516710 4091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:51.517020 4091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:51.519866 4091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:51.520013 4091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:51.520174 4091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:51.520264 4091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:51 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:51.547073 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:51.521005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.65:47719)
0806 22:26:51.521916 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:51.521953 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:51.523823 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:51.524626 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:51.524634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:51.525388 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:51.526299 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:51.526335 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.528127 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.528139 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:51.531127 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:51.531149 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.532062 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.532079 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:51.532483 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:51.533438 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:51.533487 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:51.537510 (+ 4023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:51.541755 (+ 4245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:51.541765 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:51.541782 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:51.542239 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:51.542828 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:51.542831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:51.542834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:51.543163 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:51.543980 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:51.543986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:51.544451 (+ 465us) client_negotiation.cc:769] Sending connection context
0806 22:26:51.544885 (+ 434us) client_negotiation.cc:240] Negotiation successful
0806 22:26:51.545363 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":377,"threads_started":1}
I20260806 22:26:51.548213 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:51.520349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:47719 (local address 127.1.8.126:38171)
0806 22:26:51.521272 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:51.521277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:51.522095 (+ 818us) server_negotiation.cc:407] Connection header received
0806 22:26:51.524148 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:51.524153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:51.524249 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:51.524374 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:51.526533 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.527834 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.531288 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.531720 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.531927 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:51.537846 (+ 5919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:51.537855 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:51.537863 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:51.537923 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:51.541600 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:51.542459 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:51.542468 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:51.542474 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:51.542679 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:51.543302 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:51.543310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:51.543316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:51.543795 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:51.544004 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:51.547681 (+ 3677us) server_negotiation.cc:299] Negotiation successful
0806 22:26:51.547893 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":696,"threads_started":1}
I20260806 22:26:51.572644 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:51.555383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:48545 (local address 127.1.8.126:38171)
0806 22:26:51.555634 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:51.555643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:51.556315 (+ 672us) server_negotiation.cc:407] Connection header received
0806 22:26:51.556535 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:51.556540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:51.556637 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:51.556723 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:51.558442 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.559149 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.562057 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.562474 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.562578 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:51.565643 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:51.565651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:51.565658 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:51.565705 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:51.569110 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:51.570075 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:51.570090 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:51.570094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:51.570245 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:51.570798 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:51.570807 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:51.570809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:51.571235 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:51.571408 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:51.571972 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 22:26:51.572241 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:26:51.572755 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:51.555345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.65:48545)
0806 22:26:51.556161 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:51.556190 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:51.556386 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:51.556807 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:51.556815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:51.557726 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:51.558253 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:51.558271 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.559424 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.559433 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:51.561868 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:51.561903 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:51.562705 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:51.562715 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:51.562965 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:51.563723 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:51.563755 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:51.565322 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:51.569268 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:51.569274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:51.569278 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:51.569878 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:51.570436 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:51.570445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:51.570451 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:51.570655 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:51.571380 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:51.571385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:51.571662 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 22:26:51.571846 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 22:26:51.572267 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":398,"threads_started":1}
I20260806 22:26:51.579483 4091 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:46705
I20260806 22:26:51.579506 4301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:46705 every 8 connection(s)
I20260806 22:26:51.580806 4091 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:26:51.582916 4303 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38171
I20260806 22:26:51.583168 4303 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:51.583724 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4091
I20260806 22:26:51.583901 4303 heartbeater.cc:511] Master 127.1.8.126:38171 requested a full tablet report, sending...
I20260806 22:26:51.583945 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:26:51.584656 1057 external_mini_cluster.cc:1428] Setting key ab281a312bb0a6307a86c57b6e408a01
I20260806 22:26:51.586228 3979 ts_manager.cc:194] Registered new tserver with Master: 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705)
I20260806 22:26:51.587795 3979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:47719
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:26:51.635916 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38171
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:51.979310 4313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:51.979631 4313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:51.979777 4313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:51.987021 4313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:51.987141 4313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:51.987262 4313 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:26:51.987370 4313 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:51.987493 4313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:26:51.997721 4313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:51.999157 4313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:52.000818 4313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:52.014803 4327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:52.015100 4323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:52.014835 4322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:52.015977 4313 server_base.cc:1034] running on GCE node
I20260806 22:26:52.016691 4313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:52.017729 4313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:52.018967 4313 hybrid_clock.cc:648] HybridClock initialized: now 1786055212018912 us; error 68 us; skew 500 ppm
I20260806 22:26:52.019307 4313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:52.022662 4313 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:26:52.024197 4313 webserver.cc:460] Webserver started at http://127.1.8.66:38605/ using document root <none> and password file <none>
I20260806 22:26:52.024906 4313 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:52.025063 4313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:52.025401 4313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:52.028563 4313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "ba190b39e4c84518bf228b375d0be865"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "097f0ed7668d864f3b92f6aad9bd0e9b"
server_key_iv: "ffc49fa636878c7000ff5b2ce3963cb4"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.029384 4313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "ba190b39e4c84518bf228b375d0be865"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "097f0ed7668d864f3b92f6aad9bd0e9b"
server_key_iv: "ffc49fa636878c7000ff5b2ce3963cb4"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.035355 4313 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 22:26:52.039122 4341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:52.040563 4313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:52.040748 4313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "ba190b39e4c84518bf228b375d0be865"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "097f0ed7668d864f3b92f6aad9bd0e9b"
server_key_iv: "ffc49fa636878c7000ff5b2ce3963cb4"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.040954 4313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:52.054749 4313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:52.059443 4313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:52.059800 4313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:52.062788 4313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:52.062919 4313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:52.063069 4313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:52.063180 4313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:52.089936 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.063166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:45165 (local address 127.1.8.126:38171)
0806 22:26:52.063385 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.063393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.065050 (+ 1657us) server_negotiation.cc:407] Connection header received
0806 22:26:52.067471 (+ 2421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.067481 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.067603 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.067721 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:52.070055 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.071141 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.074267 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.074997 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.075137 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.081656 (+ 6519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:52.081667 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:52.081671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:52.081722 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:52.085562 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.086673 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.086681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.086687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.086873 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.087578 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.087584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.087590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.087936 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:52.088171 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.089378 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.089629 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:26:52.091610 4354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.063840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.66:45165)
0806 22:26:52.064878 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.064925 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.067148 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.067995 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.068004 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.068859 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:52.069831 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.069862 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.071311 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.071318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.074020 (+ 2702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.074045 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.075417 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.075449 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.075854 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.076853 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:52.076886 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:52.081254 (+ 4368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:52.085736 (+ 4482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.085747 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.085769 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.086514 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.087019 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.087028 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.087032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.087400 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.088153 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:52.088162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:52.088861 (+ 699us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.089276 (+ 415us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.090442 (+ 1166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":513,"threads_started":1}
I20260806 22:26:52.113436 4313 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:37027
I20260806 22:26:52.113479 4525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:37027 every 8 connection(s)
I20260806 22:26:52.115129 4313 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:26:52.117975 4528 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38171
I20260806 22:26:52.118242 4528 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:52.119020 4528 heartbeater.cc:511] Master 127.1.8.126:38171 requested a full tablet report, sending...
I20260806 22:26:52.119678 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.103749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:55689 (local address 127.1.8.126:38171)
0806 22:26:52.103950 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.103956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.105468 (+ 1512us) server_negotiation.cc:407] Connection header received
0806 22:26:52.105717 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.105724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.105821 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.105978 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:52.106986 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.108052 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.110382 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.110869 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.111033 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.113327 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:52.113333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:52.113336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:52.113370 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:52.116448 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.117425 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.117431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.117435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.117571 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.117995 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.118003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.118006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.118467 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:52.118632 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.119062 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.119297 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:26:52.119831 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.104200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.66:55689)
0806 22:26:52.105309 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.105335 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.105537 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.106020 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.106025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.106525 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:52.106816 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.106826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.108301 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.108310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.110189 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.110206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.111284 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.111289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.111411 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.112193 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:52.112215 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:52.113113 (+ 898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:52.116611 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.116623 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.116628 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.117191 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.117689 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.117694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.117698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.117886 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.118590 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:52.118596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:52.118805 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.118967 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.119323 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":681,"threads_started":1}
I20260806 22:26:52.120316 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4313
I20260806 22:26:52.120519 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:26:52.120802 3979 ts_manager.cc:194] Registered new tserver with Master: ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027)
I20260806 22:26:52.121135 1057 external_mini_cluster.cc:1428] Setting key 235524fd4ca7ac6511b8dc80f39724b1
I20260806 22:26:52.121861 3979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:45165
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:26:52.179626 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38171
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:26:52.540416 4534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:26:52.540779 4534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:26:52.540975 4534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:26:52.548668 4534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:26:52.548842 4534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:26:52.549006 4534 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:26:52.549098 4534 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:26:52.549239 4534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:26:52.560043 4534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:26:52.561686 4534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:26:52.563198 4534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:26:52.577490 4560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:52.577694 4556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:26:52.577612 4557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:26:52.578713 4534 server_base.cc:1034] running on GCE node
I20260806 22:26:52.579509 4534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:26:52.580797 4534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:26:52.582059 4534 hybrid_clock.cc:648] HybridClock initialized: now 1786055212581994 us; error 59 us; skew 500 ppm
I20260806 22:26:52.582409 4534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:26:52.585963 4534 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:26:52.587464 4534 webserver.cc:460] Webserver started at http://127.1.8.67:38915/ using document root <none> and password file <none>
I20260806 22:26:52.588248 4534 fs_manager.cc:359] Metadata directory not provided
I20260806 22:26:52.588408 4534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:26:52.588733 4534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:26:52.590565 4303 heartbeater.cc:503] Master 127.1.8.126:38171 was elected leader, sending a full tablet report...
I20260806 22:26:52.591986 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "e0659e74a3f3409e96b4196484382fbb"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "c2fc646b9a9fea11108d1e6ddb0f5a91"
server_key_iv: "0972b0791b012032162964350072f03a"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.592957 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "e0659e74a3f3409e96b4196484382fbb"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "c2fc646b9a9fea11108d1e6ddb0f5a91"
server_key_iv: "0972b0791b012032162964350072f03a"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.598682 4534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:26:52.602289 4578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:52.603832 4534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:26:52.603968 4534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "e0659e74a3f3409e96b4196484382fbb"
format_stamp: "Formatted at 2026-08-06 22:26:52 on dist-test-slave-7vjv"
server_key: "c2fc646b9a9fea11108d1e6ddb0f5a91"
server_key_iv: "0972b0791b012032162964350072f03a"
server_key_version: "encryptionkey@0"
I20260806 22:26:52.604208 4534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:26:52.649119 4534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:26:52.654115 4534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:26:52.654482 4534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:26:52.657521 4534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:26:52.657618 4534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:26:52.657815 4534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:26:52.657915 4534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:52.682982 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.658026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:41729 (local address 127.1.8.126:38171)
0806 22:26:52.658663 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.658671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.659911 (+ 1240us) server_negotiation.cc:407] Connection header received
0806 22:26:52.662060 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.662066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.662162 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.662283 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:52.664656 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.666078 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.669512 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.670059 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.670209 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.675981 (+ 5772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:52.675986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:52.675989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:52.676025 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:52.679646 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.680494 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.680500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.680504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.680652 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.681117 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.681123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.681130 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.681403 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:52.681586 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.682306 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.682540 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":306,"threads_started":1}
I20260806 22:26:52.684221 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.658807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.67:41729)
0806 22:26:52.659719 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.659780 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.661764 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.662535 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.662544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.663369 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:52.664426 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.664465 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.666343 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.666355 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.669312 (+ 2957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.669338 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.670576 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.670585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.670924 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.671803 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:52.671826 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:52.675619 (+ 3793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:52.679804 (+ 4185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.679814 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.679831 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.680357 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.680770 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.680775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.680779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.680983 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.681539 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:52.681548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:52.681924 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.682195 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.682756 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":360,"threads_started":1}
I20260806 22:26:52.708658 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.691243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:35099 (local address 127.1.8.126:38171)
0806 22:26:52.691477 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.691487 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.692374 (+ 887us) server_negotiation.cc:407] Connection header received
0806 22:26:52.692422 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.692425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.692481 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.692601 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:52.694463 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.695722 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.698087 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.698616 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.698766 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.701792 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:52.701802 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:52.701808 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:52.701854 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:52.705297 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.706265 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.706270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.706273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.706430 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.706903 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.706906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.706908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.707221 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:52.707411 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.707935 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.708264 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:26:52.709008 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.691206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.1.8.67:35099)
0806 22:26:52.692048 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.692072 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.692281 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.692669 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.692677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.693526 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:52.694244 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.694267 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.696002 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.696007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.697912 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.697934 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.698846 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.698855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.699131 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.699875 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:52.699915 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:52.701507 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:52.705426 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.705431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.705435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.706109 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.706565 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:52.706573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:52.706579 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:52.706777 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:52.707360 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:52.707368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:52.707626 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.707807 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.708321 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":449,"threads_started":1}
I20260806 22:26:52.712788 4534 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:34319
I20260806 22:26:52.712823 4778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:34319 every 8 connection(s)
I20260806 22:26:52.714062 4534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:26:52.716151 4779 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38171
I20260806 22:26:52.716400 4779 heartbeater.cc:464] Registering TS with master...
I20260806 22:26:52.716807 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4534
I20260806 22:26:52.717003 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:26:52.717242 4779 heartbeater.cc:511] Master 127.1.8.126:38171 requested a full tablet report, sending...
I20260806 22:26:52.717555 1057 external_mini_cluster.cc:1428] Setting key e8d64e41b0b5c03b3aa73447f12570bb
I20260806 22:26:52.718878 3977 ts_manager.cc:194] Registered new tserver with Master: e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:52.720178 3977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:41729
I20260806 22:26:52.720788 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:26:52 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:52.777102 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.762401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51690 (local address 127.1.8.126:38171)
0806 22:26:52.762653 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.762660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.762881 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:26:52.763109 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.763115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.763202 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.763313 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:52.764806 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.765558 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.766882 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.767289 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.767428 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.771303 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:52.771307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:52.771311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:52.771345 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:52.774593 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.775203 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.775207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.775209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.775354 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:52.775744 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:52.775747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:52.775749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:52.776022 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:52.776316 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.776551 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.776732 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:26:52.781292 3977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:52.788461 3977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:52.812674 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.801721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.1.8.66:37027)
0806 22:26:52.802437 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.802450 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.803314 (+ 864us) server_negotiation.cc:407] Connection header received
0806 22:26:52.803789 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.803801 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.804129 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.804293 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:52.806171 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.807423 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.809983 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.811403 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.811539 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.811764 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.812067 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.812345 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":660,"threads_started":1}
I20260806 22:26:52.813010 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.801088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:37027 (local address 127.0.0.1:40822)
0806 22:26:52.803130 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.803183 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.803495 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.805581 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.805586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.805611 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:52.806027 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.806036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.807548 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.807554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.809829 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.809842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.811604 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.811611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.811801 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.811855 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.811997 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.812533 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1700,"mutex_wait_us":32,"thread_start_us":1518,"threads_started":1}
I20260806 22:26:52.814303 4810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.803993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.0.0.1:59746)
0806 22:26:52.804970 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.804991 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.805127 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.806345 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.806350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.806374 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:52.806736 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.806748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.808830 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.808837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.810925 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.810935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.813461 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.813467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.813652 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.813695 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.813797 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.813913 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":3,"thread_start_us":742,"threads_started":1}
I20260806 22:26:52.814518 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.804563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.1.8.67:34319)
0806 22:26:52.805798 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.805803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.805827 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:26:52.805939 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.805945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.806154 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.806342 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:52.806930 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.808674 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.811075 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.812417 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.813422 (+ 1005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.813641 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.813838 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.814002 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1095,"thread_start_us":889,"threads_started":1}
I20260806 22:26:52.814746 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.802237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.1.8.65:46705)
0806 22:26:52.803744 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:52.803752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:52.804372 (+ 620us) server_negotiation.cc:407] Connection header received
0806 22:26:52.804550 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:52.804557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:52.804779 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:52.804974 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:52.805921 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.807337 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.810626 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.812015 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.812245 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.812424 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:52.814140 (+ 1716us) server_negotiation.cc:299] Negotiation successful
0806 22:26:52.814319 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1361,"thread_start_us":1222,"threads_started":1}
I20260806 22:26:52.815462 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:52.801321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:46705 (local address 127.0.0.1:57382)
0806 22:26:52.804228 (+ 2907us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:52.804247 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:52.804389 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:52.805200 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:52.805207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:52.805236 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:52.805715 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.805733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.807475 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.807482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.810464 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:52.810478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:52.812580 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:52.812586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:52.812884 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:52.812928 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:26:52.813022 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:26:52.815114 (+ 2092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2734,"mutex_wait_us":125,"thread_start_us":2605,"threads_started":1}
I20260806 22:26:52.816931 4408 tablet_service.cc:1467] Processing CreateTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf (DEFAULT_TABLE table=test-table [id=0bc485516a3644a0bc09bda859cb8958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:52.818259 4408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3991cb70aa342f48cc33d3a91cd1ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:52.817843 4661 tablet_service.cc:1467] Processing CreateTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf (DEFAULT_TABLE table=test-table [id=0bc485516a3644a0bc09bda859cb8958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:52.818647 4194 tablet_service.cc:1467] Processing CreateTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf (DEFAULT_TABLE table=test-table [id=0bc485516a3644a0bc09bda859cb8958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:52.819273 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3991cb70aa342f48cc33d3a91cd1ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:52.819916 4194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3991cb70aa342f48cc33d3a91cd1ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:52.829515 4814 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Bootstrap starting.
I20260806 22:26:52.833506 4814 tablet_bootstrap.cc:654] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:52.834589 4814 log.cc:824] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:52.835934 4818 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Bootstrap starting.
I20260806 22:26:52.837653 4821 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Bootstrap starting.
I20260806 22:26:52.837925 4814 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: No bootstrap required, opened a new log
I20260806 22:26:52.838220 4814 ts_tablet_manager.cc:1397] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 22:26:52.841339 4821 tablet_bootstrap.cc:654] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:52.842545 4818 tablet_bootstrap.cc:654] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:52.842623 4821 log.cc:824] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:52.843725 4818 log.cc:824] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Log is configured to *not* fsync() on all Append() calls
I20260806 22:26:52.846006 4821 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: No bootstrap required, opened a new log
I20260806 22:26:52.846285 4821 ts_tablet_manager.cc:1397] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:26:52.846593 4818 tablet_bootstrap.cc:492] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: No bootstrap required, opened a new log
I20260806 22:26:52.846844 4818 ts_tablet_manager.cc:1397] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 22:26:52.849115 4814 raft_consensus.cc:348] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.849594 4814 raft_consensus.cc:374] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:52.849701 4814 raft_consensus.cc:729] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Initialized, Role: FOLLOWER
I20260806 22:26:52.850370 4814 consensus_queue.cc:260] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.851596 4528 heartbeater.cc:503] Master 127.1.8.126:38171 was elected leader, sending a full tablet report...
I20260806 22:26:52.853785 4814 ts_tablet_manager.cc:1428] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260806 22:26:52.856187 4821 raft_consensus.cc:348] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.856561 4821 raft_consensus.cc:374] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:52.856681 4821 raft_consensus.cc:729] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Initialized, Role: FOLLOWER
I20260806 22:26:52.856478 4818 raft_consensus.cc:348] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.856874 4818 raft_consensus.cc:374] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:52.857021 4818 raft_consensus.cc:729] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Initialized, Role: FOLLOWER
I20260806 22:26:52.857367 4821 consensus_queue.cc:260] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.857582 4818 consensus_queue.cc:260] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:52.858567 4779 heartbeater.cc:503] Master 127.1.8.126:38171 was elected leader, sending a full tablet report...
I20260806 22:26:52.859395 4821 ts_tablet_manager.cc:1428] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 22:26:52.860800 4818 ts_tablet_manager.cc:1428] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
W20260806 22:26:52.867617 4529 tablet.cc:2369] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:26:52.965906 4783 tablet.cc:2369] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:26:53.083387 4305 tablet.cc:2369] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:26:53.172420 4827 raft_consensus.cc:482] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:53.172791 4827 raft_consensus.cc:504] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.174547 4827 leader_election.cc:288] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:53.181447 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.174891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:39699 (local address 127.1.8.65:46705)
0806 22:26:53.175152 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.175159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.175910 (+ 751us) server_negotiation.cc:407] Connection header received
0806 22:26:53.176200 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.176205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.176308 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.176421 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:53.176950 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.177679 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.179584 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.180461 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.180619 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.180726 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.181080 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.181198 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:26:53.181545 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.175163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:42581 (local address 127.1.8.67:34319)
0806 22:26:53.175392 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.175399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.176351 (+ 952us) server_negotiation.cc:407] Connection header received
0806 22:26:53.176550 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.176554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.176623 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.176720 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:53.177332 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.178014 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.179762 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.180594 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.180748 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.180872 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.181166 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.181292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:26:53.181560 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.174784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:46705 (local address 127.1.8.66:39699)
0806 22:26:53.175765 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.175782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.176014 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.176441 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.176445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.176474 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:53.176778 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.176787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.177823 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.177834 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.179400 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.179413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.180683 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.180689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.180913 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.180937 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.181042 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.181164 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":109,"thread_start_us":610,"threads_started":1}
I20260806 22:26:53.181696 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.175253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.1.8.66:42581)
0806 22:26:53.176201 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.176229 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.176392 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.176745 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.176749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.176778 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:53.177175 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.177184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.178153 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.178159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.179555 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.179568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.180831 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.180837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.181005 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.181027 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.181139 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.181253 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":5,"thread_start_us":560,"threads_started":1}
I20260806 22:26:53.182695 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3991cb70aa342f48cc33d3a91cd1ccf" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb" is_pre_election: true
I20260806 22:26:53.182668 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3991cb70aa342f48cc33d3a91cd1ccf" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37" is_pre_election: true
I20260806 22:26:53.183247 4705 raft_consensus.cc:2454] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 0.
I20260806 22:26:53.183247 4226 raft_consensus.cc:2454] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 0.
I20260806 22:26:53.183985 4342 leader_election.cc:302] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, ba190b39e4c84518bf228b375d0be865; no voters:
I20260806 22:26:53.184508 4827 raft_consensus.cc:2781] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:53.184664 4827 raft_consensus.cc:482] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:53.184793 4827 raft_consensus.cc:3037] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.187304 4827 raft_consensus.cc:504] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.187904 4827 leader_election.cc:288] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 election: Requested vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:53.188481 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3991cb70aa342f48cc33d3a91cd1ccf" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37"
I20260806 22:26:53.188481 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3991cb70aa342f48cc33d3a91cd1ccf" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb"
I20260806 22:26:53.188707 4705 raft_consensus.cc:3037] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.188706 4226 raft_consensus.cc:3037] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.191869 4226 raft_consensus.cc:2454] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 1.
I20260806 22:26:53.191869 4705 raft_consensus.cc:2454] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 1.
I20260806 22:26:53.192370 4343 leader_election.cc:302] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba190b39e4c84518bf228b375d0be865, e0659e74a3f3409e96b4196484382fbb; no voters:
I20260806 22:26:53.192718 4827 raft_consensus.cc:2781] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:53.193204 4827 raft_consensus.cc:686] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 1 LEADER]: Becoming Leader. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Running, Role: LEADER
I20260806 22:26:53.193663 4827 consensus_queue.cc:237] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.198391 3977 catalog_manager.cc:5495] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba190b39e4c84518bf228b375d0be865 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "ba190b39e4c84518bf228b375d0be865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:26:53 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055210, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:53.304636 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.290357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38171 (local address 127.0.0.1:51692)
0806 22:26:53.290943 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.290963 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.291095 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.291705 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.291708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.292457 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:26:53.292717 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.292726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.293801 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.293805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.294886 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.294899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.295714 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.295719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.295845 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.296601 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:26:53.296618 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:26:53.298881 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:26:53.302215 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:53.302226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:53.302232 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:53.302726 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:53.303111 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:26:53.303114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:26:53.303116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:26:53.303253 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:26:53.303749 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:26:53.303755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:26:53.303899 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.304041 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.304312 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":281,"threads_started":1}
I20260806 22:26:53.304888 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.290373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.1.8.126:38171)
0806 22:26:53.291435 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.291440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.291459 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:26:53.291506 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.291510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.291563 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.291660 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:53.292881 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.293690 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.295088 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.295537 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.295652 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.299138 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:53.299143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:53.299146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:53.299174 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:53.302097 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:53.302845 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:53.302849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:53.302851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:53.303014 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:53.303361 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:53.303364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:53.303366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:53.303610 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:53.303738 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.304203 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.304440 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":948,"thread_start_us":844,"threads_started":1}
I20260806 22:26:53.309921 3977 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:51692:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:26:53.310886 3977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:53.317435 4194 tablet_service.cc:1467] Processing CreateTablet for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=830b6df4521146239d9e5f5a93367c15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:53.317732 4661 tablet_service.cc:1467] Processing CreateTablet for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=830b6df4521146239d9e5f5a93367c15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:53.317704 4408 tablet_service.cc:1467] Processing CreateTablet for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=830b6df4521146239d9e5f5a93367c15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:26:53.318446 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.318442 4194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.318553 4408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2f72a67e0b4ec3bdeb39b425e7aaf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.324386 4814 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: Bootstrap starting.
I20260806 22:26:53.325686 4818 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: Bootstrap starting.
I20260806 22:26:53.325845 4814 tablet_bootstrap.cc:654] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.327544 4818 tablet_bootstrap.cc:654] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.328315 4821 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: Bootstrap starting.
I20260806 22:26:53.329634 4821 tablet_bootstrap.cc:654] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.336627 4814 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: No bootstrap required, opened a new log
I20260806 22:26:53.336705 4821 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: No bootstrap required, opened a new log
I20260806 22:26:53.336792 4814 ts_tablet_manager.cc:1397] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260806 22:26:53.336828 4821 ts_tablet_manager.cc:1397] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:26:53.337518 4818 tablet_bootstrap.cc:492] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: No bootstrap required, opened a new log
I20260806 22:26:53.337685 4818 ts_tablet_manager.cc:1397] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260806 22:26:53.337754 4821 raft_consensus.cc:348] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.337761 4814 raft_consensus.cc:348] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.338049 4821 raft_consensus.cc:374] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.338040 4814 raft_consensus.cc:374] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.338168 4821 raft_consensus.cc:729] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.338167 4814 raft_consensus.cc:729] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.338441 4821 consensus_queue.cc:260] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.338474 4814 consensus_queue.cc:260] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.338506 4818 raft_consensus.cc:348] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.338711 4818 raft_consensus.cc:374] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.338804 4818 raft_consensus.cc:729] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.339047 4818 consensus_queue.cc:260] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.339144 4821 ts_tablet_manager.cc:1428] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 22:26:53.338985 4828 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.339339 4828 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.339758 4818 ts_tablet_manager.cc:1428] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 22:26:53.339574 4829 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.340142 4829 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.340730 4827 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.341050 4827 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.341620 4814 ts_tablet_manager.cc:1428] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:26:53.588253 4828 raft_consensus.cc:482] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:53.588517 4828 raft_consensus.cc:504] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.590116 4828 leader_election.cc:288] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027)
I20260806 22:26:53.597687 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.590636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:50165 (local address 127.1.8.66:37027)
0806 22:26:53.591347 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.591353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.591975 (+ 622us) server_negotiation.cc:407] Connection header received
0806 22:26:53.592186 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.592196 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.592257 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.592362 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:53.592951 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.593694 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.595786 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.596630 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.596782 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.596887 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.597153 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.597283 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":385,"threads_started":1}
I20260806 22:26:53.597723 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.590520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:37027 (local address 127.1.8.67:50165)
0806 22:26:53.591854 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.591871 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.592020 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.592378 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.592386 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.592410 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:53.592781 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.592790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.593834 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.593842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.595631 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.595648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.596798 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.596804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.596982 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.597005 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.597124 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.597250 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":81,"thread_start_us":663,"threads_started":1}
I20260806 22:26:53.597710 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.590634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:39007 (local address 127.1.8.65:46705)
0806 22:26:53.590858 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.590867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.591697 (+ 830us) server_negotiation.cc:407] Connection header received
0806 22:26:53.591890 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.591894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.591957 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.592082 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:53.592735 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.593556 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.595809 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.596673 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.596838 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.596967 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.597279 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.597413 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:26:53.597877 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.590520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:46705 (local address 127.1.8.67:39007)
0806 22:26:53.591532 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.591557 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.591741 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.592145 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.592151 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.592183 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:53.592535 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.592547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.593698 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.593707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.595643 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.595657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.596915 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.596922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.597112 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.597136 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.597250 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.597387 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":108,"thread_start_us":973,"threads_started":1}
I20260806 22:26:53.598613 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2f72a67e0b4ec3bdeb39b425e7aaf2" candidate_uuid: "e0659e74a3f3409e96b4196484382fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37" is_pre_election: true
I20260806 22:26:53.598877 4226 raft_consensus.cc:2454] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0659e74a3f3409e96b4196484382fbb in term 0.
I20260806 22:26:53.598929 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2f72a67e0b4ec3bdeb39b425e7aaf2" candidate_uuid: "e0659e74a3f3409e96b4196484382fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865" is_pre_election: true
I20260806 22:26:53.599381 4579 leader_election.cc:302] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, e0659e74a3f3409e96b4196484382fbb; no voters:
I20260806 22:26:53.599464 4448 raft_consensus.cc:2454] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0659e74a3f3409e96b4196484382fbb in term 0.
I20260806 22:26:53.599784 4828 raft_consensus.cc:2781] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:53.599907 4828 raft_consensus.cc:482] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:53.600000 4828 raft_consensus.cc:3037] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.602232 4828 raft_consensus.cc:504] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.602900 4828 leader_election.cc:288] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [CANDIDATE]: Term 1 election: Requested vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027)
I20260806 22:26:53.603364 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2f72a67e0b4ec3bdeb39b425e7aaf2" candidate_uuid: "e0659e74a3f3409e96b4196484382fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37"
I20260806 22:26:53.603470 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2f72a67e0b4ec3bdeb39b425e7aaf2" candidate_uuid: "e0659e74a3f3409e96b4196484382fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865"
I20260806 22:26:53.603608 4226 raft_consensus.cc:3037] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.603715 4448 raft_consensus.cc:3037] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:53.605859 4226 raft_consensus.cc:2454] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0659e74a3f3409e96b4196484382fbb in term 1.
I20260806 22:26:53.605926 4448 raft_consensus.cc:2454] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0659e74a3f3409e96b4196484382fbb in term 1.
I20260806 22:26:53.606280 4579 leader_election.cc:302] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, e0659e74a3f3409e96b4196484382fbb; no voters:
I20260806 22:26:53.606614 4828 raft_consensus.cc:2781] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:53.607065 4828 raft_consensus.cc:686] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [term 1 LEADER]: Becoming Leader. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Running, Role: LEADER
I20260806 22:26:53.607609 4828 consensus_queue.cc:237] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.611918 4854 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: TxnStatusTablet state changed. Reason: New leader e0659e74a3f3409e96b4196484382fbb. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.612355 4854 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:53.612550 3977 catalog_manager.cc:5495] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb reported cstate change: term changed from 0 to 1, leader changed from <none> to e0659e74a3f3409e96b4196484382fbb (127.1.8.67). New cstate: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } health_report { overall_health: HEALTHY } } }
I20260806 22:26:53.613184 4858 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:26:53.613444 4858 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:26:53.697618 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.691315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.1.8.67:34319)
0806 22:26:53.691913 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.691917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.691934 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:26:53.692020 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.692024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.692186 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.692315 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:53.692768 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.693508 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.695001 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.695484 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.696171 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.696268 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.696881 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.697064 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.697169 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.697257 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
I20260806 22:26:53.697592 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.691216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.0.0.1:59756)
0806 22:26:53.691467 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.691491 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.691645 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.692372 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.692377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.692391 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:53.692627 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.692639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.693648 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.693657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.694854 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.694867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.695753 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.695759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.695887 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.695947 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.697008 (+ 1061us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.697031 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.697128 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.697292 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:26:53.706084 4448 raft_consensus.cc:1261] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Refusing update from remote peer e0659e74a3f3409e96b4196484382fbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:53.706197 4226 raft_consensus.cc:1261] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Refusing update from remote peer e0659e74a3f3409e96b4196484382fbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:53.707188 4828 consensus_queue.cc:1035] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:53.707968 4828 consensus_queue.cc:1035] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:53.735668 4829 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: TxnStatusTablet state changed. Reason: New leader e0659e74a3f3409e96b4196484382fbb. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.736001 4829 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.737433 4827 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: TxnStatusTablet state changed. Reason: New leader e0659e74a3f3409e96b4196484382fbb. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.737749 4827 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.739709 4865 mvcc.cc:205] Tried to move back new op lower bound from 7315682155329355776 to 7315682154941431808. Current Snapshot: MvccSnapshot[applied={T|T < 7315682155329355776}]
I20260806 22:26:53.741336 4827 consensus_queue.cc:1035] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:53.742043 4868 mvcc.cc:205] Tried to move back new op lower bound from 7315682155329355776 to 7315682154941431808. Current Snapshot: MvccSnapshot[applied={T|T < 7315682155329355776}]
I20260806 22:26:53.743801 4827 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.744056 4827 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P ba190b39e4c84518bf228b375d0be865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.746958 4872 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.747413 4872 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:53.746958 4828 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.747686 4828 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P e0659e74a3f3409e96b4196484382fbb: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:26:53.758560 4869 consensus_queue.cc:1035] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260806 22:26:53.764518 4829 tablet_replica.cc:439] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0659e74a3f3409e96b4196484382fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } } }
I20260806 22:26:53.764780 4829 tablet_replica.cc:442] T dd2f72a67e0b4ec3bdeb39b425e7aaf2 P 655077bd63d94cb6b6351feb9ffdfb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:26:53.771867 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.757685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.1.8.66:37027)
0806 22:26:53.757910 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.757918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.757943 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:26:53.758062 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.758070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.758198 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.758377 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:53.766168 (+ 7791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.767094 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.769640 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.770067 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.770216 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.770696 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.771219 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.771384 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.771477 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.771575 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:26:53.789644 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.783466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:43475 (local address 127.1.8.67:34319)
0806 22:26:53.783658 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.783663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.784323 (+ 660us) server_negotiation.cc:407] Connection header received
0806 22:26:53.784559 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.784564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.784639 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.784740 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:53.785232 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.785990 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.787621 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.788084 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.788262 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.788749 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.788908 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.789041 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.789184 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.789264 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:26:53.789686 4897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.783435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.1.8.66:43475)
0806 22:26:53.784186 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.784204 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.784404 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.784756 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.784761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.784772 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:53.785073 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.785082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.786141 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.786150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.787452 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.787466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.788319 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.788326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.788461 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.788535 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.789044 (+ 509us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.789059 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.789161 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.789307 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":418,"threads_started":1}
I20260806 22:26:53.831664 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.824126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:40209 (local address 127.1.8.66:37027)
0806 22:26:53.824331 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:53.824339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:53.825240 (+ 901us) server_negotiation.cc:407] Connection header received
0806 22:26:53.825458 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:53.825463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:53.825558 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:53.825656 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:53.826766 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.827707 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.829250 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.829875 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.829995 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.830650 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.830790 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.830889 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:53.831268 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 22:26:53.831358 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:26:53.831681 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:53.824180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:37027 (local address 127.1.8.67:40209)
0806 22:26:53.825116 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:53.825133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:53.825316 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:53.826042 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:53.826046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:53.826056 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:53.826606 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.826618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.827833 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.827840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.829107 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:53.829120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:53.830069 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:53.830079 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:53.830334 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:53.830416 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:53.830891 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:53.830926 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:26:53.831039 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:26:53.831196 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":529,"threads_started":1}
I20260806 22:26:53.885846 3976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:53.886183 3976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:53.890689 3976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Sending DeleteTablet for 3 replicas of tablet c3991cb70aa342f48cc33d3a91cd1ccf
I20260806 22:26:53.892241 4408 tablet_service.cc:1514] Processing DeleteTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:53 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40822
I20260806 22:26:53.892267 4194 tablet_service.cc:1514] Processing DeleteTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:53 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260806 22:26:53.893692 4661 tablet_service.cc:1514] Processing DeleteTablet for tablet c3991cb70aa342f48cc33d3a91cd1ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:53 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:59746
I20260806 22:26:53.898573 3976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:26:53.900799 4910 tablet_replica.cc:330] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: stopping tablet replica
W20260806 22:26:53.900869 3976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:53.902045 4910 raft_consensus.cc:2229] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:53.903676 4910 raft_consensus.cc:2258] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:53.912230 4910 ts_tablet_manager.cc:1905] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:53.916976 4193 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (DEFAULT_TABLE table=test-table [id=8ef37d3021104fd5a0122096bd8b7585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:53.917557 4193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.919113 4661 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (DEFAULT_TABLE table=test-table [id=8ef37d3021104fd5a0122096bd8b7585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:53.919574 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.920722 4407 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (DEFAULT_TABLE table=test-table [id=8ef37d3021104fd5a0122096bd8b7585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:53.921350 4407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:53.921769 4909 tablet_replica.cc:330] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: stopping tablet replica
I20260806 22:26:53.922255 4909 raft_consensus.cc:2229] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:53.922670 4909 raft_consensus.cc:2258] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:53.924866 4909 ts_tablet_manager.cc:1905] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:53.932262 4910 ts_tablet_manager.cc:1918] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:53.932574 4910 log.cc:1196] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/c3991cb70aa342f48cc33d3a91cd1ccf
I20260806 22:26:53.933331 4910 ts_tablet_manager.cc:1939] T c3991cb70aa342f48cc33d3a91cd1ccf P e0659e74a3f3409e96b4196484382fbb: Deleting consensus metadata
I20260806 22:26:53.937990 4909 ts_tablet_manager.cc:1918] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:53.940147 3957 catalog_manager.cc:4841] TS e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319): tablet c3991cb70aa342f48cc33d3a91cd1ccf (table test-table [id=0bc485516a3644a0bc09bda859cb8958]) successfully deleted
I20260806 22:26:53.940304 4909 log.cc:1196] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/c3991cb70aa342f48cc33d3a91cd1ccf
I20260806 22:26:53.940899 4909 ts_tablet_manager.cc:1939] T c3991cb70aa342f48cc33d3a91cd1ccf P 655077bd63d94cb6b6351feb9ffdfb37: Deleting consensus metadata
I20260806 22:26:53.942929 3955 catalog_manager.cc:4841] TS 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705): tablet c3991cb70aa342f48cc33d3a91cd1ccf (table test-table [id=0bc485516a3644a0bc09bda859cb8958]) successfully deleted
I20260806 22:26:53.944442 4913 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Bootstrap starting.
I20260806 22:26:53.946535 4913 tablet_bootstrap.cc:654] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.950948 4913 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: No bootstrap required, opened a new log
I20260806 22:26:53.951318 4913 ts_tablet_manager.cc:1397] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260806 22:26:53.952313 4913 raft_consensus.cc:348] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.952700 4913 raft_consensus.cc:374] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.952855 4913 raft_consensus.cc:729] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.953266 4913 consensus_queue.cc:260] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.957731 4913 ts_tablet_manager.cc:1428] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:26:53.964612 4907 tablet_replica.cc:330] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: stopping tablet replica
I20260806 22:26:53.966749 4911 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Bootstrap starting.
I20260806 22:26:53.967445 4912 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Bootstrap starting.
I20260806 22:26:53.967834 4907 raft_consensus.cc:2229] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:53.968715 4907 raft_consensus.cc:2258] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:53.969867 4911 tablet_bootstrap.cc:654] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.970925 4912 tablet_bootstrap.cc:654] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:53.974170 4907 ts_tablet_manager.cc:1905] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:53.984942 4911 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: No bootstrap required, opened a new log
I20260806 22:26:53.984902 4907 ts_tablet_manager.cc:1918] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:53.985116 4907 log.cc:1196] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/c3991cb70aa342f48cc33d3a91cd1ccf
I20260806 22:26:53.985244 4911 ts_tablet_manager.cc:1397] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260806 22:26:53.985834 4907 ts_tablet_manager.cc:1939] T c3991cb70aa342f48cc33d3a91cd1ccf P ba190b39e4c84518bf228b375d0be865: Deleting consensus metadata
I20260806 22:26:53.986171 4911 raft_consensus.cc:348] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.986505 4911 raft_consensus.cc:374] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.986658 4911 raft_consensus.cc:729] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.987049 4911 consensus_queue.cc:260] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.987941 3957 catalog_manager.cc:4841] TS ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027): tablet c3991cb70aa342f48cc33d3a91cd1ccf (table test-table [id=0bc485516a3644a0bc09bda859cb8958]) successfully deleted
I20260806 22:26:53.989681 4911 ts_tablet_manager.cc:1428] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:26:53.993337 4912 tablet_bootstrap.cc:492] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: No bootstrap required, opened a new log
I20260806 22:26:53.993513 4912 ts_tablet_manager.cc:1397] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.000s
I20260806 22:26:53.994427 4912 raft_consensus.cc:348] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.994719 4912 raft_consensus.cc:374] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:53.994815 4912 raft_consensus.cc:729] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Initialized, Role: FOLLOWER
I20260806 22:26:53.995226 4912 consensus_queue.cc:260] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:53.995931 4912 ts_tablet_manager.cc:1428] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:26:54.104791 4829 raft_consensus.cc:482] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:54.105074 4829 raft_consensus.cc:504] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:54.106925 4829 leader_election.cc:288] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:54.115084 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.107266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:38555 (local address 127.1.8.66:37027)
0806 22:26:54.107537 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:54.107546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:54.108112 (+ 566us) server_negotiation.cc:407] Connection header received
0806 22:26:54.108399 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:54.108406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:54.108492 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:54.108582 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:54.109396 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.110600 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.112491 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.113884 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.113978 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.114090 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:54.114664 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 22:26:54.114803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:26:54.115757 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.107249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:37027 (local address 127.1.8.65:38555)
0806 22:26:54.107977 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:54.107998 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:54.108169 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:54.108858 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:54.108862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:54.108905 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:54.109188 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.109217 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.110757 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.110764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.112271 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.112305 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.114223 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.114229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.114444 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.114480 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:26:54.115268 (+ 788us) client_negotiation.cc:240] Negotiation successful
0806 22:26:54.115412 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":499,"threads_started":1}
I20260806 22:26:54.116030 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.108365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:46243 (local address 127.1.8.67:34319)
0806 22:26:54.108533 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:54.108542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:54.109095 (+ 553us) server_negotiation.cc:407] Connection header received
0806 22:26:54.109302 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:54.109310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:54.109403 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:54.109512 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:26:54.110009 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.110665 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.113256 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.114521 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.114633 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.114733 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:54.115606 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 22:26:54.115767 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:26:54.116726 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7b76f7dfaf44c39a5a33f1209a82ac" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865" is_pre_election: true
I20260806 22:26:54.116995 4448 raft_consensus.cc:2454] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 0.
I20260806 22:26:54.117141 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.108173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.1.8.65:46243)
0806 22:26:54.108971 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:54.108989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:54.109142 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:54.109527 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:54.109532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:54.109554 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:26:54.109859 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.109866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.110815 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.110821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.113064 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.113080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.114913 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.114919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.115094 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.115132 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:26:54.116611 (+ 1479us) client_negotiation.cc:240] Negotiation successful
0806 22:26:54.116770 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":407,"threads_started":1}
I20260806 22:26:54.117959 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7b76f7dfaf44c39a5a33f1209a82ac" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb" is_pre_election: true
I20260806 22:26:54.118300 4705 raft_consensus.cc:2454] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 0.
I20260806 22:26:54.118276 4119 leader_election.cc:302] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, ba190b39e4c84518bf228b375d0be865; no voters:
I20260806 22:26:54.118752 4829 raft_consensus.cc:2781] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:54.118883 4829 raft_consensus.cc:482] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:54.118988 4829 raft_consensus.cc:3037] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:54.122193 4829 raft_consensus.cc:504] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:54.122889 4829 leader_election.cc:288] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 election: Requested vote from peers ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:54.123449 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7b76f7dfaf44c39a5a33f1209a82ac" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb"
I20260806 22:26:54.123505 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7b76f7dfaf44c39a5a33f1209a82ac" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865"
I20260806 22:26:54.123695 4705 raft_consensus.cc:3037] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:54.123732 4448 raft_consensus.cc:3037] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:54.126709 4705 raft_consensus.cc:2454] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 1.
I20260806 22:26:54.127135 4448 raft_consensus.cc:2454] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 1.
I20260806 22:26:54.127228 4119 leader_election.cc:302] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, e0659e74a3f3409e96b4196484382fbb; no voters:
I20260806 22:26:54.127722 4829 raft_consensus.cc:2781] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:54.127936 4829 raft_consensus.cc:686] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 LEADER]: Becoming Leader. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Running, Role: LEADER
I20260806 22:26:54.128432 4829 consensus_queue.cc:237] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:54.132658 3977 catalog_manager.cc:5495] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "655077bd63d94cb6b6351feb9ffdfb37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } health_report { overall_health: UNKNOWN } } }
I20260806 22:26:54.188359 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.179663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.1.8.65:46705)
0806 22:26:54.180374 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:54.180378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:54.180404 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:26:54.180486 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:54.180489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:54.180559 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:54.180755 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:54.181245 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.182751 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.184894 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.185677 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.186406 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.186591 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:54.187353 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:54.187554 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:54.187745 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:26:54.187839 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":458,"threads_started":1}
I20260806 22:26:54.215906 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.205816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44437 (local address 127.1.8.67:34319)
0806 22:26:54.206036 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:54.206044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:54.207113 (+ 1069us) server_negotiation.cc:407] Connection header received
0806 22:26:54.207425 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:54.207433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:54.207540 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:54.207637 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:54.208555 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.210048 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.213093 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.213971 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.214120 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.214956 (+ 836us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:54.215134 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:54.215226 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:54.215436 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:26:54.215535 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:26:54.216476 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.205924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:34319 (local address 127.1.8.65:44437)
0806 22:26:54.206988 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:54.207009 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:54.207257 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:54.207718 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:54.207723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:54.207738 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:54.208336 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.208352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.210275 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.210282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.212915 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.212934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.214258 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.214267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.214565 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.214707 (+ 142us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:54.215262 (+ 555us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:54.215286 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:26:54.215387 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:26:54.215554 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":612,"threads_started":1}
I20260806 22:26:54.226624 4705 raft_consensus.cc:1261] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Refusing update from remote peer 655077bd63d94cb6b6351feb9ffdfb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:54.227459 4448 raft_consensus.cc:1261] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Refusing update from remote peer 655077bd63d94cb6b6351feb9ffdfb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:54.227569 4885 consensus_queue.cc:1035] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 22:26:54.232223 4829 consensus_queue.cc:1035] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:54.238621 4866 mvcc.cc:205] Tried to move back new op lower bound from 7315682157464289280 to 7315682157074759680. Current Snapshot: MvccSnapshot[applied={T|T < 7315682157464289280}]
I20260806 22:26:54.271445 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.263750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:46705 (local address 127.1.8.67:54599)
0806 22:26:54.263993 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:26:54.264016 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:26:54.264177 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:26:54.264780 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:26:54.264785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:26:54.264797 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:26:54.265125 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.265141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.267043 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.267048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.268705 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:26:54.268723 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.270118 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.270126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:26:54.270327 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.270394 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:26:54.270899 (+ 505us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:26:54.270910 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:26:54.270980 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:26:54.271105 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:26:54.271536 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:54.264062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:54599 (local address 127.1.8.65:46705)
0806 22:26:54.264266 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:54.264272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:54.264302 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:26:54.264382 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:54.264387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:54.264471 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:54.264562 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:54.265296 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.266888 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.268869 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:54.269810 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:54.269961 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:54.270628 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:54.270765 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:54.270883 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:54.271010 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:26:54.271111 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 22:26:54.273196 4866 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:26:55.904899 3977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:55.905234 3977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:55.910408 3977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Sending DeleteTablet for 3 replicas of tablet 9c7b76f7dfaf44c39a5a33f1209a82ac
I20260806 22:26:55.911418 4407 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:55 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40822
I20260806 22:26:55.911597 4193 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:55 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260806 22:26:55.916684 4661 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7b76f7dfaf44c39a5a33f1209a82ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:55 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:59746
I20260806 22:26:55.933265 4976 tablet_replica.cc:330] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: stopping tablet replica
I20260806 22:26:55.933646 4976 raft_consensus.cc:2229] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:55.934589 4976 raft_consensus.cc:2258] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:55.936046 4976 ts_tablet_manager.cc:1905] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:55.938789 4977 tablet_replica.cc:330] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: stopping tablet replica
I20260806 22:26:55.939193 4977 raft_consensus.cc:2229] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:55.939590 4977 raft_consensus.cc:2258] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:55.940729 4977 ts_tablet_manager.cc:1905] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:55.944366 4976 ts_tablet_manager.cc:1918] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:55.944792 4974 tablet_replica.cc:330] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: stopping tablet replica
I20260806 22:26:55.945053 4974 raft_consensus.cc:2229] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:55.945452 4974 raft_consensus.cc:2258] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:55.946753 4976 log.cc:1196] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/9c7b76f7dfaf44c39a5a33f1209a82ac
I20260806 22:26:55.947237 4976 ts_tablet_manager.cc:1939] T 9c7b76f7dfaf44c39a5a33f1209a82ac P 655077bd63d94cb6b6351feb9ffdfb37: Deleting consensus metadata
I20260806 22:26:55.948426 3955 catalog_manager.cc:4841] TS 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705): tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (table test-table [id=8ef37d3021104fd5a0122096bd8b7585]) successfully deleted
I20260806 22:26:55.948863 4974 ts_tablet_manager.cc:1905] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:55.950414 4977 ts_tablet_manager.cc:1918] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:55.950618 4977 log.cc:1196] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/9c7b76f7dfaf44c39a5a33f1209a82ac
I20260806 22:26:55.951295 4977 ts_tablet_manager.cc:1939] T 9c7b76f7dfaf44c39a5a33f1209a82ac P e0659e74a3f3409e96b4196484382fbb: Deleting consensus metadata
I20260806 22:26:55.952585 3957 catalog_manager.cc:4841] TS e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319): tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (table test-table [id=8ef37d3021104fd5a0122096bd8b7585]) successfully deleted
I20260806 22:26:55.954710 4974 ts_tablet_manager.cc:1918] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:55.954955 4974 log.cc:1196] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/9c7b76f7dfaf44c39a5a33f1209a82ac
I20260806 22:26:55.955504 4974 ts_tablet_manager.cc:1939] T 9c7b76f7dfaf44c39a5a33f1209a82ac P ba190b39e4c84518bf228b375d0be865: Deleting consensus metadata
I20260806 22:26:55.956580 3957 catalog_manager.cc:4841] TS ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027): tablet 9c7b76f7dfaf44c39a5a33f1209a82ac (table test-table [id=8ef37d3021104fd5a0122096bd8b7585]) successfully deleted
Aug 06 22:26:55 dist-test-slave-7vjv krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055215, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:26:55 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055215, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:26:55.996300 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:55.980817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.1.8.126:38171)
0806 22:26:55.981457 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:55.981465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:55.981582 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:26:55.981812 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:55.981816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:55.981885 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:55.982003 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:55.983382 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:55.984325 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:55.985719 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:55.986187 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:55.986295 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:55.990595 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:55.990599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:55.990603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:55.990636 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:55.993616 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:55.994285 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:55.994289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:55.994291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:55.994419 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:55.994795 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:55.994797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:55.994799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:55.995194 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:55.995415 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:55.995658 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 22:26:55.995848 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":293,"threads_started":1}
I20260806 22:26:55.999959 3977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:56.000804 3977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:56.007613 4193 tablet_service.cc:1467] Processing CreateTablet for tablet 9aad051b42284287b7a2e9e63884ade4 (DEFAULT_TABLE table=test-table [id=a68538605f5f451c8c4b62a6282be7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.007963 4407 tablet_service.cc:1467] Processing CreateTablet for tablet 9aad051b42284287b7a2e9e63884ade4 (DEFAULT_TABLE table=test-table [id=a68538605f5f451c8c4b62a6282be7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.008172 4193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aad051b42284287b7a2e9e63884ade4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.008433 4661 tablet_service.cc:1467] Processing CreateTablet for tablet 9aad051b42284287b7a2e9e63884ade4 (DEFAULT_TABLE table=test-table [id=a68538605f5f451c8c4b62a6282be7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.008630 4407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aad051b42284287b7a2e9e63884ade4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.008821 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aad051b42284287b7a2e9e63884ade4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.015558 4999 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Bootstrap starting.
I20260806 22:26:56.016637 5000 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Bootstrap starting.
I20260806 22:26:56.019222 5000 tablet_bootstrap.cc:654] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.019258 4999 tablet_bootstrap.cc:654] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.021549 5002 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Bootstrap starting.
I20260806 22:26:56.024689 5002 tablet_bootstrap.cc:654] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.025876 4999 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: No bootstrap required, opened a new log
I20260806 22:26:56.026055 4999 ts_tablet_manager.cc:1397] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260806 22:26:56.027043 5000 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: No bootstrap required, opened a new log
I20260806 22:26:56.027058 4999 raft_consensus.cc:348] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.027295 5000 ts_tablet_manager.cc:1397] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260806 22:26:56.027407 4999 raft_consensus.cc:374] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.027604 4999 raft_consensus.cc:729] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.027956 4999 consensus_queue.cc:260] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.028472 5000 raft_consensus.cc:348] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.028808 4999 ts_tablet_manager.cc:1428] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:26:56.028934 5000 raft_consensus.cc:374] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.029124 5000 raft_consensus.cc:729] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.029560 5000 consensus_queue.cc:260] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.030236 5000 ts_tablet_manager.cc:1428] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 22:26:56.030833 5002 tablet_bootstrap.cc:492] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: No bootstrap required, opened a new log
I20260806 22:26:56.031073 5002 ts_tablet_manager.cc:1397] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 22:26:56.032328 5002 raft_consensus.cc:348] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.032655 5002 raft_consensus.cc:374] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.032866 5002 raft_consensus.cc:729] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.033180 5002 consensus_queue.cc:260] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.034189 5002 ts_tablet_manager.cc:1428] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:26:56.311647 4955 raft_consensus.cc:482] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:56.311939 4955 raft_consensus.cc:504] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.312793 4955 leader_election.cc:288] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:56.313485 4447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aad051b42284287b7a2e9e63884ade4" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865" is_pre_election: true
I20260806 22:26:56.313535 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aad051b42284287b7a2e9e63884ade4" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb" is_pre_election: true
I20260806 22:26:56.313796 4447 raft_consensus.cc:2454] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 0.
I20260806 22:26:56.313890 4705 raft_consensus.cc:2454] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 0.
I20260806 22:26:56.314435 4119 leader_election.cc:302] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, ba190b39e4c84518bf228b375d0be865; no voters:
I20260806 22:26:56.314788 4955 raft_consensus.cc:2781] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:56.314954 4955 raft_consensus.cc:482] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:56.315078 4955 raft_consensus.cc:3037] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.317750 4955 raft_consensus.cc:504] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.318399 4955 leader_election.cc:288] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 election: Requested vote from peers ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:56.319064 4447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aad051b42284287b7a2e9e63884ade4" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba190b39e4c84518bf228b375d0be865"
I20260806 22:26:56.319113 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aad051b42284287b7a2e9e63884ade4" candidate_uuid: "655077bd63d94cb6b6351feb9ffdfb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb"
I20260806 22:26:56.319442 4447 raft_consensus.cc:3037] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.319551 4705 raft_consensus.cc:3037] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.323544 4705 raft_consensus.cc:2454] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 1.
I20260806 22:26:56.323558 4447 raft_consensus.cc:2454] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 655077bd63d94cb6b6351feb9ffdfb37 in term 1.
I20260806 22:26:56.324079 4119 leader_election.cc:302] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, ba190b39e4c84518bf228b375d0be865; no voters:
I20260806 22:26:56.324512 4955 raft_consensus.cc:2781] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:56.324712 4955 raft_consensus.cc:686] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 LEADER]: Becoming Leader. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Running, Role: LEADER
I20260806 22:26:56.325158 4955 consensus_queue.cc:237] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.328737 3977 catalog_manager.cc:5495] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "655077bd63d94cb6b6351feb9ffdfb37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } health_report { overall_health: UNKNOWN } } }
I20260806 22:26:56.397307 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:56.390304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.1.8.65:46705)
0806 22:26:56.391151 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:56.391155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:56.391170 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:26:56.391223 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:56.391227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:56.391287 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:56.391399 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:56.392081 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:56.393147 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:56.394966 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:56.395646 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:56.395756 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:56.396377 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:56.396518 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:56.396591 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:56.396750 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:26:56.396817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":617,"threads_started":1}
I20260806 22:26:56.407402 4447 raft_consensus.cc:1261] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Refusing update from remote peer 655077bd63d94cb6b6351feb9ffdfb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:56.407488 4705 raft_consensus.cc:1261] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Refusing update from remote peer 655077bd63d94cb6b6351feb9ffdfb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:56.408146 4964 consensus_queue.cc:1035] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:56.408890 4955 consensus_queue.cc:1035] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:56.477566 3976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:56.478047 3976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:56.482553 3976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Sending DeleteTablet for 3 replicas of tablet 9aad051b42284287b7a2e9e63884ade4
I20260806 22:26:56.483677 4193 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aad051b42284287b7a2e9e63884ade4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:56 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260806 22:26:56.483740 4661 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aad051b42284287b7a2e9e63884ade4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:56 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:59746
I20260806 22:26:56.483819 4407 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aad051b42284287b7a2e9e63884ade4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:56 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40822
I20260806 22:26:56.484731 5028 tablet_replica.cc:330] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: stopping tablet replica
I20260806 22:26:56.485076 5028 raft_consensus.cc:2229] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:56.485657 5028 raft_consensus.cc:2258] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:56.487402 5028 ts_tablet_manager.cc:1905] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:56.487205 3976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:26:56.488160 3976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:26:56.496372 5031 tablet_replica.cc:330] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: stopping tablet replica
I20260806 22:26:56.497741 5028 ts_tablet_manager.cc:1918] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:56.497829 5031 raft_consensus.cc:2229] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:56.498077 5028 log.cc:1196] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/9aad051b42284287b7a2e9e63884ade4
I20260806 22:26:56.498153 5031 raft_consensus.cc:2258] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:56.499114 5028 ts_tablet_manager.cc:1939] T 9aad051b42284287b7a2e9e63884ade4 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting consensus metadata
I20260806 22:26:56.499042 4408 tablet_service.cc:1467] Processing CreateTablet for tablet 45158cf633ea40ae8c94e23066d14845 (DEFAULT_TABLE table=test-table [id=159c79cd46ca42888c69d735e45c1704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.499432 4193 tablet_service.cc:1467] Processing CreateTablet for tablet 45158cf633ea40ae8c94e23066d14845 (DEFAULT_TABLE table=test-table [id=159c79cd46ca42888c69d735e45c1704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.499433 4661 tablet_service.cc:1467] Processing CreateTablet for tablet 45158cf633ea40ae8c94e23066d14845 (DEFAULT_TABLE table=test-table [id=159c79cd46ca42888c69d735e45c1704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:26:56.499560 4408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45158cf633ea40ae8c94e23066d14845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.499857 4193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45158cf633ea40ae8c94e23066d14845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.499871 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45158cf633ea40ae8c94e23066d14845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:26:56.505496 5034 tablet_replica.cc:330] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: stopping tablet replica
I20260806 22:26:56.505827 5034 raft_consensus.cc:2229] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:56.505960 5002 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Bootstrap starting.
I20260806 22:26:56.506183 5034 raft_consensus.cc:2258] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:56.507531 5000 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Bootstrap starting.
I20260806 22:26:56.508466 5002 tablet_bootstrap.cc:654] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.508613 5034 ts_tablet_manager.cc:1905] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:56.509796 3955 catalog_manager.cc:4841] TS 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705): tablet 9aad051b42284287b7a2e9e63884ade4 (table test-table [id=a68538605f5f451c8c4b62a6282be7de]) successfully deleted
I20260806 22:26:56.509940 5000 tablet_bootstrap.cc:654] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.511435 5031 ts_tablet_manager.cc:1905] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:56.512533 4999 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Bootstrap starting.
I20260806 22:26:56.515527 4999 tablet_bootstrap.cc:654] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Neither blocks nor log segments found. Creating new log.
I20260806 22:26:56.519197 5002 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: No bootstrap required, opened a new log
I20260806 22:26:56.519651 5002 ts_tablet_manager.cc:1397] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260806 22:26:56.521406 5002 raft_consensus.cc:348] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.521935 5002 raft_consensus.cc:374] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.522177 5002 raft_consensus.cc:729] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.522672 5002 consensus_queue.cc:260] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.524634 5002 ts_tablet_manager.cc:1428] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 22:26:56.529184 5000 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: No bootstrap required, opened a new log
I20260806 22:26:56.529408 5031 ts_tablet_manager.cc:1918] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:56.529515 5000 ts_tablet_manager.cc:1397] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.006s
I20260806 22:26:56.529579 5031 log.cc:1196] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/9aad051b42284287b7a2e9e63884ade4
I20260806 22:26:56.530113 5034 ts_tablet_manager.cc:1918] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:26:56.530339 5034 log.cc:1196] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/9aad051b42284287b7a2e9e63884ade4
I20260806 22:26:56.530437 5031 ts_tablet_manager.cc:1939] T 9aad051b42284287b7a2e9e63884ade4 P e0659e74a3f3409e96b4196484382fbb: Deleting consensus metadata
I20260806 22:26:56.530680 5000 raft_consensus.cc:348] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.530958 5034 ts_tablet_manager.cc:1939] T 9aad051b42284287b7a2e9e63884ade4 P ba190b39e4c84518bf228b375d0be865: Deleting consensus metadata
I20260806 22:26:56.530973 5000 raft_consensus.cc:374] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.531102 5000 raft_consensus.cc:729] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655077bd63d94cb6b6351feb9ffdfb37, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.531450 5000 consensus_queue.cc:260] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.531720 3957 catalog_manager.cc:4841] TS e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319): tablet 9aad051b42284287b7a2e9e63884ade4 (table test-table [id=a68538605f5f451c8c4b62a6282be7de]) successfully deleted
I20260806 22:26:56.532114 3957 catalog_manager.cc:4841] TS ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027): tablet 9aad051b42284287b7a2e9e63884ade4 (table test-table [id=a68538605f5f451c8c4b62a6282be7de]) successfully deleted
I20260806 22:26:56.532485 5000 ts_tablet_manager.cc:1428] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:26:56.533901 4999 tablet_bootstrap.cc:492] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: No bootstrap required, opened a new log
I20260806 22:26:56.534093 4999 ts_tablet_manager.cc:1397] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.008s
I20260806 22:26:56.535061 4999 raft_consensus.cc:348] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.535279 4999 raft_consensus.cc:374] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:26:56.535359 4999 raft_consensus.cc:729] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0659e74a3f3409e96b4196484382fbb, State: Initialized, Role: FOLLOWER
I20260806 22:26:56.535564 4999 consensus_queue.cc:260] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.536149 4999 ts_tablet_manager.cc:1428] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 22:26:56.610074 5012 raft_consensus.cc:482] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:26:56.610411 5012 raft_consensus.cc:504] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.611430 5012 leader_election.cc:288] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:56.611953 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45158cf633ea40ae8c94e23066d14845" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37" is_pre_election: true
I20260806 22:26:56.612011 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45158cf633ea40ae8c94e23066d14845" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb" is_pre_election: true
I20260806 22:26:56.612363 4226 raft_consensus.cc:2454] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 0.
I20260806 22:26:56.612346 4705 raft_consensus.cc:2454] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 0.
I20260806 22:26:56.612800 4343 leader_election.cc:302] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba190b39e4c84518bf228b375d0be865, e0659e74a3f3409e96b4196484382fbb; no voters:
I20260806 22:26:56.613129 5012 raft_consensus.cc:2781] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:26:56.613346 5012 raft_consensus.cc:482] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:26:56.613487 5012 raft_consensus.cc:3037] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.616459 5012 raft_consensus.cc:504] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.617273 5012 leader_election.cc:288] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 election: Requested vote from peers 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705), e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319)
I20260806 22:26:56.617569 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45158cf633ea40ae8c94e23066d14845" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655077bd63d94cb6b6351feb9ffdfb37"
I20260806 22:26:56.617865 4226 raft_consensus.cc:3037] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.617893 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45158cf633ea40ae8c94e23066d14845" candidate_uuid: "ba190b39e4c84518bf228b375d0be865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0659e74a3f3409e96b4196484382fbb"
I20260806 22:26:56.618333 4705 raft_consensus.cc:3037] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:26:56.620460 4226 raft_consensus.cc:2454] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 1.
I20260806 22:26:56.620884 4342 leader_election.cc:302] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655077bd63d94cb6b6351feb9ffdfb37, ba190b39e4c84518bf228b375d0be865; no voters:
I20260806 22:26:56.621269 5012 raft_consensus.cc:2781] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:26:56.621359 4705 raft_consensus.cc:2454] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba190b39e4c84518bf228b375d0be865 in term 1.
I20260806 22:26:56.622088 5012 raft_consensus.cc:686] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 1 LEADER]: Becoming Leader. State: Replica: ba190b39e4c84518bf228b375d0be865, State: Running, Role: LEADER
I20260806 22:26:56.622486 5012 consensus_queue.cc:237] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } }
I20260806 22:26:56.625401 3977 catalog_manager.cc:5495] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba190b39e4c84518bf228b375d0be865 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "ba190b39e4c84518bf228b375d0be865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba190b39e4c84518bf228b375d0be865" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 37027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 } health_report { overall_health: UNKNOWN } } }
I20260806 22:26:56.660354 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:56.653625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.1.8.66:37027)
0806 22:26:56.654730 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:56.654735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:56.654752 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:26:56.654799 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:56.654802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:56.654855 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:56.655019 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:26:56.655375 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:56.656694 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:56.658066 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:56.658666 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:56.658785 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:56.659363 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:26:56.659472 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:26:56.659553 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:56.659722 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:26:56.659784 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":721,"threads_started":1}
I20260806 22:26:56.669767 4226 raft_consensus.cc:1261] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Refusing update from remote peer ba190b39e4c84518bf228b375d0be865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:56.669884 4705 raft_consensus.cc:1261] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Refusing update from remote peer ba190b39e4c84518bf228b375d0be865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:26:56.670435 5040 consensus_queue.cc:1035] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655077bd63d94cb6b6351feb9ffdfb37" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:26:56.670872 5012 consensus_queue.cc:1035] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0659e74a3f3409e96b4196484382fbb" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:26:56.694154 4941 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:26:58.285059 3976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:58.285342 3976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:26:58.291108 3976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b9440962d84d819b4595a03f02ba9f: Sending DeleteTablet for 3 replicas of tablet 45158cf633ea40ae8c94e23066d14845
I20260806 22:26:58.292160 4193 tablet_service.cc:1514] Processing DeleteTablet for tablet 45158cf633ea40ae8c94e23066d14845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:58 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260806 22:26:58.292237 4408 tablet_service.cc:1514] Processing DeleteTablet for tablet 45158cf633ea40ae8c94e23066d14845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:58 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40822
I20260806 22:26:58.292716 4659 tablet_service.cc:1514] Processing DeleteTablet for tablet 45158cf633ea40ae8c94e23066d14845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:26:58 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:59746
I20260806 22:26:58.295315 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4091
I20260806 22:26:58.302038 5094 tablet_replica.cc:330] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: stopping tablet replica
I20260806 22:26:58.303534 5094 raft_consensus.cc:2229] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:58.304538 5094 raft_consensus.cc:2258] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:58.306524 5094 ts_tablet_manager.cc:1905] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:58.308192 5089 tablet_replica.cc:330] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: stopping tablet replica
I20260806 22:26:58.308487 5089 raft_consensus.cc:2229] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:26:58.309008 5089 raft_consensus.cc:2258] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:58.309949 5090 tablet_replica.cc:330] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: stopping tablet replica
I20260806 22:26:58.310278 5090 raft_consensus.cc:2229] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:26:58.310719 5089 ts_tablet_manager.cc:1905] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:58.310895 5090 raft_consensus.cc:2258] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:26:58.312126 5090 ts_tablet_manager.cc:1905] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:26:58.317327 5094 ts_tablet_manager.cc:1918] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:58.317636 5094 log.cc:1196] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/45158cf633ea40ae8c94e23066d14845
I20260806 22:26:58.318523 5094 ts_tablet_manager.cc:1939] T 45158cf633ea40ae8c94e23066d14845 P e0659e74a3f3409e96b4196484382fbb: Deleting consensus metadata
I20260806 22:26:58.319080 5089 ts_tablet_manager.cc:1918] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:58.319286 5089 log.cc:1196] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/45158cf633ea40ae8c94e23066d14845
I20260806 22:26:58.319908 5089 ts_tablet_manager.cc:1939] T 45158cf633ea40ae8c94e23066d14845 P 655077bd63d94cb6b6351feb9ffdfb37: Deleting consensus metadata
I20260806 22:26:58.320057 3957 catalog_manager.cc:4841] TS e0659e74a3f3409e96b4196484382fbb (127.1.8.67:34319): tablet 45158cf633ea40ae8c94e23066d14845 (table test-table [id=159c79cd46ca42888c69d735e45c1704]) successfully deleted
I20260806 22:26:58.320521 5090 ts_tablet_manager.cc:1918] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:26:58.320672 5090 log.cc:1196] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/45158cf633ea40ae8c94e23066d14845
I20260806 22:26:58.321138 3955 catalog_manager.cc:4841] TS 655077bd63d94cb6b6351feb9ffdfb37 (127.1.8.65:46705): tablet 45158cf633ea40ae8c94e23066d14845 (table test-table [id=159c79cd46ca42888c69d735e45c1704]) successfully deleted
I20260806 22:26:58.321307 5090 ts_tablet_manager.cc:1939] T 45158cf633ea40ae8c94e23066d14845 P ba190b39e4c84518bf228b375d0be865: Deleting consensus metadata
I20260806 22:26:58.322234 3957 catalog_manager.cc:4841] TS ba190b39e4c84518bf228b375d0be865 (127.1.8.66:37027): tablet 45158cf633ea40ae8c94e23066d14845 (table test-table [id=159c79cd46ca42888c69d735e45c1704]) successfully deleted
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055215, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:26:58.337316 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:58.298499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.1.8.65:46705)
0806 22:26:58.309504 (+ 11005us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:58.309513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:58.310302 (+ 789us) server_negotiation.cc:407] Connection header received
0806 22:26:58.310372 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:58.310378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:58.310459 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:58.310564 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:58.313290 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.314698 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.317258 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.317995 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.318112 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:58.328930 (+ 10818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:58.328964 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:58.328983 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:58.329025 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:58.333662 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.334415 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.334429 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.334431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.334620 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.335177 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.335181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.335183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.335600 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:58.336148 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:58.336360 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:26:58.336614 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10831,"thread_start_us":10978,"threads_started":1}
W20260806 22:26:58.338512 4297 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57408
W20260806 22:26:58.339274 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4091: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:58.339619 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4091
I20260806 22:26:58.378515 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4313
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055215, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:26:58.395381 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:58.379350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.1.8.66:37027)
0806 22:26:58.380053 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:58.380057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:58.380074 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:26:58.380187 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:58.380193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:58.380271 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:58.380365 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:58.382191 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.382949 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.384549 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.384995 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.385102 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:58.389539 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:58.389554 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:58.389573 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:58.389608 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:58.392763 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.393377 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.393384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.393386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.393529 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.393830 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.393834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.393836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.394271 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:58.394518 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:58.394849 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:26:58.395022 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":472,"threads_started":1}
W20260806 22:26:58.396075 4519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40842
W20260806 22:26:58.396797 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4313: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:58.396967 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4313
I20260806 22:26:58.431120 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4534
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055215, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:26:58.446714 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:26:58.431902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.1.8.67:34319)
0806 22:26:58.432548 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 22:26:58.432553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:26:58.432589 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:26:58.432673 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:26:58.432681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:26:58.432783 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:26:58.432926 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:26:58.434448 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.435199 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.436456 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:26:58.436885 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:26:58.436976 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:26:58.441224 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:26:58.441240 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:26:58.441257 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:26:58.441298 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:26:58.444391 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.444984 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.444991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.444993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.445132 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:26:58.445449 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:26:58.445452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:26:58.445454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:26:58.445817 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:26:58.446054 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 22:26:58.446272 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:26:58.446445 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":328,"threads_started":1}
W20260806 22:26:58.447618 4771 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59772
W20260806 22:26:58.448410 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4534: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:26:58.448619 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4534
I20260806 22:26:58.479940 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3931
I20260806 22:26:58.481356 4040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:26:58.602952 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3931
2026-08-06T22:26:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9758 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[5106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[5106](info): set up 2 sockets
Aug 06 22:26:58 dist-test-slave-7vjv krb5kdc[5106](info): commencing operation
krb5kdc: starting...
W20260806 22:26:59.953142 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:00 dist-test-slave-7vjv krb5kdc[5106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:00.194077 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37965
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:37965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:00.542125 5124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:00.542444 5124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:00.542614 5124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:00.549906 5124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:00.550022 5124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:00.550132 5124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:00.550247 5124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:00.550341 5124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:00.560637 5124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:37965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:00.562019 5124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:00.563383 5124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:00.576071 5130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:00.576156 5133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:00.576231 5131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:00.576637 5124 server_base.cc:1034] running on GCE node
I20260806 22:27:00.577769 5124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:00.579237 5124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:00.580447 5124 hybrid_clock.cc:648] HybridClock initialized: now 1786055220580394 us; error 32 us; skew 500 ppm
I20260806 22:27:00.580781 5124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:00 dist-test-slave-7vjv krb5kdc[5106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:00.583990 5124 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:00.585551 5124 webserver.cc:460] Webserver started at http://127.1.8.126:33407/ using document root <none> and password file <none>
I20260806 22:27:00.586236 5124 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:00.586340 5124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:00.586694 5124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:00.589763 5124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "c70a02168da34650ab2cf91816c81fce"
format_stamp: "Formatted at 2026-08-06 22:27:00 on dist-test-slave-7vjv"
server_key: "725901d7289a9dcdbee860f8d2329ff0"
server_key_iv: "084b4091b2f7f03905c36b948c39da16"
server_key_version: "encryptionkey@0"
I20260806 22:27:00.590548 5124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "c70a02168da34650ab2cf91816c81fce"
format_stamp: "Formatted at 2026-08-06 22:27:00 on dist-test-slave-7vjv"
server_key: "725901d7289a9dcdbee860f8d2329ff0"
server_key_iv: "084b4091b2f7f03905c36b948c39da16"
server_key_version: "encryptionkey@0"
I20260806 22:27:00.596065 5124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 22:27:00.599943 5146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:00.601395 5124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 22:27:00.601591 5124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "c70a02168da34650ab2cf91816c81fce"
format_stamp: "Formatted at 2026-08-06 22:27:00 on dist-test-slave-7vjv"
server_key: "725901d7289a9dcdbee860f8d2329ff0"
server_key_iv: "084b4091b2f7f03905c36b948c39da16"
server_key_version: "encryptionkey@0"
I20260806 22:27:00.601822 5124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:00.636947 5124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:00.641170 5124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:00.641498 5124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:00.660774 5124 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:37965
I20260806 22:27:00.660831 5227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:37965 every 8 connection(s)
I20260806 22:27:00.662319 5124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:00.666311 5228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:00.668395 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5124
I20260806 22:27:00.668675 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:00.669477 1057 external_mini_cluster.cc:1428] Setting key 58732bfd02b0b7e794c24ad2f818b5da
I20260806 22:27:00.679556 5228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce: Bootstrap starting.
I20260806 22:27:00.683859 5228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce: Neither blocks nor log segments found. Creating new log.
Aug 06 22:27:00 dist-test-slave-7vjv krb5kdc[5106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:00.685169 5228 log.cc:824] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:00.688560 5228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce: No bootstrap required, opened a new log
I20260806 22:27:00.693653 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:00.671660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.1.8.126:37965)
0806 22:27:00.672573 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:00.672585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:00.672629 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:27:00.674347 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:00.674368 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:00.674755 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:00.675067 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:00.676923 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:00.678260 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:00.680176 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:00.680867 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:00.681008 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:00.685996 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:00.686006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:00.686021 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:00.686069 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:00.689306 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:00.690058 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:00.690065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:00.690072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:00.690264 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:00.690688 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:00.690694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:00.690697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:00.691271 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:00.691622 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:00.692081 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 22:27:00.692567 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":451,"threads_started":1}
I20260806 22:27:00.696771 5228 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } }
I20260806 22:27:00.697158 5228 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:00.697268 5228 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70a02168da34650ab2cf91816c81fce, State: Initialized, Role: FOLLOWER
I20260806 22:27:00.697803 5228 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } }
I20260806 22:27:00.698057 5228 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:00.698216 5228 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:00.698390 5228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:00.700836 5228 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } }
I20260806 22:27:00.701486 5228 leader_election.cc:302] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70a02168da34650ab2cf91816c81fce; no voters:
I20260806 22:27:00.702632 5228 leader_election.cc:288] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:00.702786 5234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:00.703620 5234 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [term 1 LEADER]: Becoming Leader. State: Replica: c70a02168da34650ab2cf91816c81fce, State: Running, Role: LEADER
I20260806 22:27:00.704274 5234 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } }
I20260806 22:27:00.705088 5228 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:00.707495 5235 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70a02168da34650ab2cf91816c81fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } } }
I20260806 22:27:00.707441 5237 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70a02168da34650ab2cf91816c81fce. Latest consensus state: current_term: 1 leader_uuid: "c70a02168da34650ab2cf91816c81fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a02168da34650ab2cf91816c81fce" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37965 } } }
I20260806 22:27:00.707945 5235 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:00.707986 5237 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:00.708901 5250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:00.715492 5250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:00.725179 5250 catalog_manager.cc:1347] Generated new cluster ID: e01ef0b492f64fa9adf7657b2b2a02dd
I20260806 22:27:00.725317 5250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:00.736718 5250 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:00.738579 5250 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:00.750833 5250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70a02168da34650ab2cf91816c81fce: Generated new TSK 0
I20260806 22:27:00.751915 5250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:00.807439 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:01.155946 5273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:01.156284 5273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:01.156386 5273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:01.163955 5273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:01.164124 5273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:01.164248 5273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:01.175158 5273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:01.176527 5273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:01.177809 5273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:01.191269 5282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:01.191315 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:01.191339 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:01.192214 5273 server_base.cc:1034] running on GCE node
I20260806 22:27:01.192792 5273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:01.193817 5273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:01.195024 5273 hybrid_clock.cc:648] HybridClock initialized: now 1786055221194990 us; error 38 us; skew 500 ppm
I20260806 22:27:01.195358 5273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:01 dist-test-slave-7vjv krb5kdc[5106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:01.198704 5273 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:01.200424 5273 webserver.cc:460] Webserver started at http://127.1.8.65:44593/ using document root <none> and password file <none>
I20260806 22:27:01.201159 5273 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:01.201300 5273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:01.201658 5273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:01.204833 5273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "cddfe48d5c2748a89135b8ab796f6125"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eda61ad8447ea6782ae5800593283ab6"
server_key_iv: "3ddb7bba25c95d85247e0692d3ca512f"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.205713 5273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "cddfe48d5c2748a89135b8ab796f6125"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eda61ad8447ea6782ae5800593283ab6"
server_key_iv: "3ddb7bba25c95d85247e0692d3ca512f"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.211544 5273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 22:27:01.215615 5295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.217118 5273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:01.217310 5273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "cddfe48d5c2748a89135b8ab796f6125"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eda61ad8447ea6782ae5800593283ab6"
server_key_iv: "3ddb7bba25c95d85247e0692d3ca512f"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.217555 5273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:01.231118 5273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:01.235097 5273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:01.235399 5273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:01.236402 5273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:01.237712 5273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:01.237784 5273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.237934 5273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:01.238004 5273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.272346 5273 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:38257
I20260806 22:27:01.272397 5473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:38257 every 8 connection(s)
I20260806 22:27:01.273660 5273 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:01.282481 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5273
I20260806 22:27:01.282727 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:01.283485 1057 external_mini_cluster.cc:1428] Setting key c78c30f26e548c5200cfaa2fb902109c
Aug 06 22:27:01 dist-test-slave-7vjv krb5kdc[5106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:01.294150 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:01.276340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:35633 (local address 127.1.8.126:37965)
0806 22:27:01.276800 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:01.276805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:01.277778 (+ 973us) server_negotiation.cc:407] Connection header received
0806 22:27:01.279593 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:01.279612 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:01.279704 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:01.279828 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:01.281352 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.282205 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.283998 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.284636 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.284796 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:01.288310 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:01.288315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:01.288319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:01.288349 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:01.291001 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:01.291747 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:01.291752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:01.291754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:01.291873 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:01.292391 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:01.292398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:01.292406 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:01.292664 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:01.292821 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:01.293551 (+ 730us) server_negotiation.cc:299] Negotiation successful
0806 22:27:01.293778 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":221,"threads_started":1}
I20260806 22:27:01.295130 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:01.276898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:37965 (local address 127.1.8.65:35633)
0806 22:27:01.277616 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:01.277653 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:01.279318 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:01.280070 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:01.280078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:01.280631 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:01.281179 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:01.281195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.282393 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.282397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:01.283838 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:01.283851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.284882 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.284892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:01.285156 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:01.285847 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:01.285871 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:01.288023 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:01.291152 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:01.291161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:01.291178 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:01.291608 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:01.291987 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:01.292000 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:01.292003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:01.292270 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:01.292803 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:01.292811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:01.293169 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 22:27:01.293426 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 22:27:01.293948 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":295,"threads_started":1}
I20260806 22:27:01.296823 5475 heartbeater.cc:347] Connected to a master server at 127.1.8.126:37965
I20260806 22:27:01.297179 5475 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:01.298136 5475 heartbeater.cc:511] Master 127.1.8.126:37965 requested a full tablet report, sending...
I20260806 22:27:01.300629 5170 ts_manager.cc:194] Registered new tserver with Master: cddfe48d5c2748a89135b8ab796f6125 (127.1.8.65:38257)
I20260806 22:27:01.302124 5170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:35633
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:27:01.339321 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:01.678737 5483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:01.679025 5483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:01.679205 5483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:01.686444 5483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:01.686553 5483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:01.686733 5483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:01.696305 5483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:01.697549 5483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:01.698886 5483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:01.712572 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:01.712656 5492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:01.712680 5499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:01.713789 5483 server_base.cc:1034] running on GCE node
I20260806 22:27:01.714418 5483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:01.715415 5483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:01.716650 5483 hybrid_clock.cc:648] HybridClock initialized: now 1786055221716598 us; error 45 us; skew 500 ppm
I20260806 22:27:01.716974 5483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:01 dist-test-slave-7vjv krb5kdc[5106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:01.720402 5483 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:01.722051 5483 webserver.cc:460] Webserver started at http://127.1.8.66:38245/ using document root <none> and password file <none>
I20260806 22:27:01.722853 5483 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:01.723009 5483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:01.723345 5483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:01.726504 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "a3d9adb3569a4299bd83ec5d19bcfd72"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eb7bc58aff2d96d53490c2c879133c2d"
server_key_iv: "a7bb5a8308db5400688dcf50ce5d478a"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.727316 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "a3d9adb3569a4299bd83ec5d19bcfd72"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eb7bc58aff2d96d53490c2c879133c2d"
server_key_iv: "a7bb5a8308db5400688dcf50ce5d478a"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.732848 5483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:27:01.736411 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.737919 5483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:01.738121 5483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "a3d9adb3569a4299bd83ec5d19bcfd72"
format_stamp: "Formatted at 2026-08-06 22:27:01 on dist-test-slave-7vjv"
server_key: "eb7bc58aff2d96d53490c2c879133c2d"
server_key_iv: "a7bb5a8308db5400688dcf50ce5d478a"
server_key_version: "encryptionkey@0"
I20260806 22:27:01.738332 5483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:01.764926 5483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:01.768999 5483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:01.769312 5483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:01.770354 5483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:01.771737 5483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:01.771847 5483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.772007 5483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:01.772082 5483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:01.806039 5483 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:42647
I20260806 22:27:01.806061 5682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:42647 every 8 connection(s)
I20260806 22:27:01.807389 5483 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:01.811026 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5483
I20260806 22:27:01.811235 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:01.811765 1057 external_mini_cluster.cc:1428] Setting key c151efa0d507bcff1ebae8e253391607
Aug 06 22:27:01 dist-test-slave-7vjv krb5kdc[5106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:01.829579 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:01.809941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:37215 (local address 127.1.8.126:37965)
0806 22:27:01.810423 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:01.810427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:01.813609 (+ 3182us) server_negotiation.cc:407] Connection header received
0806 22:27:01.813666 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:01.813670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:01.813731 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:01.813834 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:01.815456 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.816250 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.818418 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.818863 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.819004 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:01.822673 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:01.822681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:01.822686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:01.822724 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:01.826262 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:01.826971 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:01.826979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:01.826986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:01.827158 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:01.827554 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:01.827562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:01.827568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:01.827943 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:01.828144 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:01.828803 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 22:27:01.829091 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
I20260806 22:27:01.830243 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:01.810398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:37965 (local address 127.1.8.66:37215)
0806 22:27:01.811209 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:01.811255 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:01.813282 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:01.814216 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:01.814224 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:01.814723 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:01.815284 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:01.815300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.816415 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.816420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:01.818264 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:01.818282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:01.819108 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:01.819115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:01.819336 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:01.820033 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:01.820055 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:01.822392 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:01.826409 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:01.826416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:01.826428 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:01.826830 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:01.827268 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:01.827271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:01.827273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:01.827444 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:01.828076 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:01.828087 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:01.828428 (+ 341us) client_negotiation.cc:769] Sending connection context
0806 22:27:01.828656 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 22:27:01.829017 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":345,"threads_started":1}
I20260806 22:27:01.831825 5685 heartbeater.cc:347] Connected to a master server at 127.1.8.126:37965
I20260806 22:27:01.832196 5685 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:01.832917 5685 heartbeater.cc:511] Master 127.1.8.126:37965 requested a full tablet report, sending...
I20260806 22:27:01.834478 5170 ts_manager.cc:194] Registered new tserver with Master: a3d9adb3569a4299bd83ec5d19bcfd72 (127.1.8.66:42647)
I20260806 22:27:01.835695 5170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:37215
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:27:01.865653 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:02.204249 5693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:02.204535 5693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:02.204635 5693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:02.211846 5693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:02.211961 5693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:02.212074 5693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:02.222347 5693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:37965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:02.223719 5693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:02.225086 5693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:02.238354 5700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:02.238451 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:02.238391 5699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:02.239354 5693 server_base.cc:1034] running on GCE node
I20260806 22:27:02.239948 5693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:02.240967 5693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:02.242184 5693 hybrid_clock.cc:648] HybridClock initialized: now 1786055222242142 us; error 34 us; skew 500 ppm
I20260806 22:27:02.242515 5693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:02 dist-test-slave-7vjv krb5kdc[5106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:02.245923 5693 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:02.247419 5693 webserver.cc:460] Webserver started at http://127.1.8.67:46819/ using document root <none> and password file <none>
I20260806 22:27:02.248152 5693 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:02.248313 5693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:02.248726 5693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:02.251713 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "073e11df41414bc9aa7f40b96021c916"
format_stamp: "Formatted at 2026-08-06 22:27:02 on dist-test-slave-7vjv"
server_key: "e2d2f708a3cac553f456a336f15e2134"
server_key_iv: "509ebda89878f8efb213094fd1f95817"
server_key_version: "encryptionkey@0"
I20260806 22:27:02.252547 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "073e11df41414bc9aa7f40b96021c916"
format_stamp: "Formatted at 2026-08-06 22:27:02 on dist-test-slave-7vjv"
server_key: "e2d2f708a3cac553f456a336f15e2134"
server_key_iv: "509ebda89878f8efb213094fd1f95817"
server_key_version: "encryptionkey@0"
I20260806 22:27:02.258157 5693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:02.261754 5711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:02.263124 5693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:27:02.263311 5693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "073e11df41414bc9aa7f40b96021c916"
format_stamp: "Formatted at 2026-08-06 22:27:02 on dist-test-slave-7vjv"
server_key: "e2d2f708a3cac553f456a336f15e2134"
server_key_iv: "509ebda89878f8efb213094fd1f95817"
server_key_version: "encryptionkey@0"
I20260806 22:27:02.263521 5693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:02.282974 5693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:02.287433 5693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:02.287781 5693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:02.288903 5693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:02.290237 5693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:02.290339 5693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:02.290498 5693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:02.290573 5693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:02.306257 5475 heartbeater.cc:503] Master 127.1.8.126:37965 was elected leader, sending a full tablet report...
I20260806 22:27:02.326458 5693 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:43933
I20260806 22:27:02.326548 5875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:43933 every 8 connection(s)
I20260806 22:27:02.327771 5693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:02.328850 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5693
I20260806 22:27:02.329088 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:02.329633 1057 external_mini_cluster.cc:1428] Setting key c8f8dd2289e0ef79de7c891cdb740b1e
Aug 06 22:27:02 dist-test-slave-7vjv krb5kdc[5106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:02.352730 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:02.330748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:55579 (local address 127.1.8.126:37965)
0806 22:27:02.331388 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:02.331395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:02.333521 (+ 2126us) server_negotiation.cc:407] Connection header received
0806 22:27:02.335676 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:02.335682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:02.335777 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:02.335894 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:02.338264 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:02.339662 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:02.342208 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:02.342920 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:02.343125 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:02.347013 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:02.347019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:02.347023 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:02.347066 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:02.349684 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:02.350371 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:02.350377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:02.350381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:02.350507 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:02.350973 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:02.350978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:02.350981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:02.351258 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:02.351414 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:02.352070 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 22:27:02.352319 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":384,"threads_started":1}
I20260806 22:27:02.353636 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:02.331442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:37965 (local address 127.1.8.67:55579)
0806 22:27:02.333338 (+ 1896us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:02.333412 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:02.335330 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:02.336220 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:02.336238 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:02.337149 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:02.338053 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:02.338076 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:02.339876 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:02.339882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:02.342030 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:02.342049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:02.343226 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:02.343234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:02.343472 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:02.344306 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:02.344337 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:02.346798 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:02.349818 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:02.349826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:02.349842 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:02.350245 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:02.350636 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:02.350641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:02.350644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:02.350831 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:02.351386 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:02.351393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:02.351716 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 22:27:02.351973 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 22:27:02.352505 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"thread_start_us":1350,"threads_started":1}
I20260806 22:27:02.354935 5877 heartbeater.cc:347] Connected to a master server at 127.1.8.126:37965
I20260806 22:27:02.355247 5877 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:02.355954 5877 heartbeater.cc:511] Master 127.1.8.126:37965 requested a full tablet report, sending...
I20260806 22:27:02.357424 5170 ts_manager.cc:194] Registered new tserver with Master: 073e11df41414bc9aa7f40b96021c916 (127.1.8.67:43933)
I20260806 22:27:02.358248 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:27:02.358309 5170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:55579
W20260806 22:27:02.399917 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:02.397307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.1.8.126:37965)
0806 22:27:02.397517 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:02.397526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:02.397925 (+ 399us) server_negotiation.cc:407] Connection header received
0806 22:27:02.398152 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:02.398157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:02.398223 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:02.398334 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:02.399713 (+ 1379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60810: BlockingRecv error: recv got EOF from 127.0.0.1:60810 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:27:02.403254 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5273
W20260806 22:27:02.406301 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5273: Not authorized: Client connection negotiation failed: client connection to 127.1.8.65:38257: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:27:02.406469 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5273
I20260806 22:27:02.427331 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5483
W20260806 22:27:02.430207 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5483: Not authorized: Client connection negotiation failed: client connection to 127.1.8.66:42647: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:27:02.430394 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5483
I20260806 22:27:02.453655 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5693
W20260806 22:27:02.456682 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5693: Not authorized: Client connection negotiation failed: client connection to 127.1.8.67:43933: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:27:02.456836 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5693
I20260806 22:27:02.480280 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5124
I20260806 22:27:02.481477 5221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:02.586217 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5124
2026-08-06T22:27:02Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3977 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:02 dist-test-slave-7vjv krb5kdc[5906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:02 dist-test-slave-7vjv krb5kdc[5906](info): set up 2 sockets
Aug 06 22:27:02 dist-test-slave-7vjv krb5kdc[5906](info): commencing operation
krb5kdc: starting...
W20260806 22:27:03.895450 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[5906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:04.135783 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33575
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:04.477309 5925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:04.477600 5925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:04.477764 5925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:04.485112 5925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:04.485225 5925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:04.485350 5925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:04.485463 5925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:04.485554 5925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:04.495347 5925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:04.496764 5925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:04.498086 5925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:04.509815 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:04.509922 5938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:04.509961 5936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:04.511133 5925 server_base.cc:1034] running on GCE node
I20260806 22:27:04.511957 5925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:04.513396 5925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:04.514603 5925 hybrid_clock.cc:648] HybridClock initialized: now 1786055224514557 us; error 35 us; skew 500 ppm
I20260806 22:27:04.514916 5925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[5906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055224, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:04.522571 5925 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:04.524224 5925 webserver.cc:460] Webserver started at http://127.1.8.126:32799/ using document root <none> and password file <none>
I20260806 22:27:04.524930 5925 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:04.525081 5925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:04.525410 5925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:04.528507 5925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "0eefc646060e4b23afc6e0b682326040"
format_stamp: "Formatted at 2026-08-06 22:27:04 on dist-test-slave-7vjv"
server_key: "c3ea99f0f04799de6f4433366acb7942"
server_key_iv: "b29c0cd7695b03abb1aa73f225f0c6d2"
server_key_version: "encryptionkey@0"
I20260806 22:27:04.529290 5925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "0eefc646060e4b23afc6e0b682326040"
format_stamp: "Formatted at 2026-08-06 22:27:04 on dist-test-slave-7vjv"
server_key: "c3ea99f0f04799de6f4433366acb7942"
server_key_iv: "b29c0cd7695b03abb1aa73f225f0c6d2"
server_key_version: "encryptionkey@0"
I20260806 22:27:04.534487 5925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:04.537902 5955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:04.539209 5925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:27:04.539397 5925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "0eefc646060e4b23afc6e0b682326040"
format_stamp: "Formatted at 2026-08-06 22:27:04 on dist-test-slave-7vjv"
server_key: "c3ea99f0f04799de6f4433366acb7942"
server_key_iv: "b29c0cd7695b03abb1aa73f225f0c6d2"
server_key_version: "encryptionkey@0"
I20260806 22:27:04.539614 5925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:04.558354 5925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:04.562397 5925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:04.562695 5925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:04.581583 5925 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:33575
I20260806 22:27:04.581631 6034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:33575 every 8 connection(s)
I20260806 22:27:04.582962 5925 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:04.586615 6036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:04.590476 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5925
I20260806 22:27:04.590704 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:04.591529 1057 external_mini_cluster.cc:1428] Setting key e9c0b3dada6db3f4456e191c40e15368
I20260806 22:27:04.600148 6036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040: Bootstrap starting.
I20260806 22:27:04.604018 6036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:04.605425 6036 log.cc:824] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[5906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:04.608919 6036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040: No bootstrap required, opened a new log
I20260806 22:27:04.616112 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:04.593487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.1.8.126:33575)
0806 22:27:04.594246 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:04.594257 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:04.594299 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:27:04.595903 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:04.595924 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:04.596404 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:04.596736 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:04.598551 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:04.599834 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:04.601608 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:04.602261 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:04.602414 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:04.607323 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:04.607333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:04.607351 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:04.607410 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:04.611489 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:04.612259 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:04.612267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:04.612276 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:04.612499 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:04.612854 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:04.612859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:04.612862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:04.613625 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:04.613948 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:04.614229 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:27:04.614695 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":350,"threads_started":1}
I20260806 22:27:04.619747 6036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } }
I20260806 22:27:04.620239 6036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:04.620276 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:27:04.620361 6036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eefc646060e4b23afc6e0b682326040, State: Initialized, Role: FOLLOWER
I20260806 22:27:04.620965 6036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } }
I20260806 22:27:04.621210 6036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:04.621345 6036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:04.621513 6036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:04.623649 6036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } }
I20260806 22:27:04.624207 6036 leader_election.cc:302] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eefc646060e4b23afc6e0b682326040; no voters:
I20260806 22:27:04.624864 6036 leader_election.cc:288] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:04.625036 6043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:04.626942 6036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:04.630650 6043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [term 1 LEADER]: Becoming Leader. State: Replica: 0eefc646060e4b23afc6e0b682326040, State: Running, Role: LEADER
I20260806 22:27:04.631345 6043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } }
I20260806 22:27:04.634362 6044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eefc646060e4b23afc6e0b682326040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } } }
I20260806 22:27:04.634735 6044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:04.635241 6047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eefc646060e4b23afc6e0b682326040. Latest consensus state: current_term: 1 leader_uuid: "0eefc646060e4b23afc6e0b682326040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eefc646060e4b23afc6e0b682326040" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33575 } } }
I20260806 22:27:04.635520 6047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:04.640723 6059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:04.647049 6059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:04.657716 6059 catalog_manager.cc:1347] Generated new cluster ID: 288630042e6c4b8db9d5f390bb729f02
I20260806 22:27:04.657828 6059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:04.678875 6059 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:04.680641 6059 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:04.688185 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:04.678581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.1.8.126:33575)
0806 22:27:04.678784 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:04.678790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:04.679746 (+ 956us) server_negotiation.cc:407] Connection header received
0806 22:27:04.679937 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:04.679943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:04.680020 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:04.680156 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:04.681494 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:04.682575 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:04.684496 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:04.685151 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:04.685265 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:04.686539 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:04.686545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:04.686549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:04.687305 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:04.687374 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:04.687405 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:04.687525 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:04.687674 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:27:04.687791 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:27:04.696813 6059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eefc646060e4b23afc6e0b682326040: Generated new TSK 0
I20260806 22:27:04.697644 6059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:27:04.708961 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5925
I20260806 22:27:04.710361 6027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:04.803682 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5925
2026-08-06T22:27:04Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2212 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[6086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[6086](info): set up 2 sockets
Aug 06 22:27:04 dist-test-slave-7vjv krb5kdc[6086](info): commencing operation
krb5kdc: starting...
W20260806 22:27:06.108948 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:06 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:06.340725 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:36933
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:32807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:06.697741 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:06.698071 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:06.698240 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:06.705477 6105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:06.705626 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:06.705750 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:06.705847 6105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:06.705935 6105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:06.715636 6105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:32807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:06.716967 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:06.718274 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:06.730439 6118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:06.730482 6123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:06.730531 6119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:06.731293 6105 server_base.cc:1034] running on GCE node
I20260806 22:27:06.732074 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:06.733536 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:06.734789 6105 hybrid_clock.cc:648] HybridClock initialized: now 1786055226734746 us; error 53 us; skew 500 ppm
I20260806 22:27:06.735154 6105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:06 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:06.738372 6105 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:06.739811 6105 webserver.cc:460] Webserver started at http://127.1.8.126:45155/ using document root <none> and password file <none>
I20260806 22:27:06.740536 6105 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:06.740641 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:06.740897 6105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:06.743851 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "230908391a0749b8a1e3c235bab9268f"
format_stamp: "Formatted at 2026-08-06 22:27:06 on dist-test-slave-7vjv"
server_key: "1f0da8615bc136b7c8500396df3c9e8b"
server_key_iv: "b8198f2fa6f170ef471e9ba898a66476"
server_key_version: "encryptionkey@0"
I20260806 22:27:06.744656 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "230908391a0749b8a1e3c235bab9268f"
format_stamp: "Formatted at 2026-08-06 22:27:06 on dist-test-slave-7vjv"
server_key: "1f0da8615bc136b7c8500396df3c9e8b"
server_key_iv: "b8198f2fa6f170ef471e9ba898a66476"
server_key_version: "encryptionkey@0"
I20260806 22:27:06.749918 6105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:06.753309 6136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:06.754650 6105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:06.754801 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "230908391a0749b8a1e3c235bab9268f"
format_stamp: "Formatted at 2026-08-06 22:27:06 on dist-test-slave-7vjv"
server_key: "1f0da8615bc136b7c8500396df3c9e8b"
server_key_iv: "b8198f2fa6f170ef471e9ba898a66476"
server_key_version: "encryptionkey@0"
I20260806 22:27:06.754966 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:06.788731 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:06.792794 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:06.793049 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:06.811810 6105 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:36933
I20260806 22:27:06.811843 6217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:36933 every 8 connection(s)
I20260806 22:27:06.813277 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:06.814468 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 6105
I20260806 22:27:06.814721 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:06.815577 1057 external_mini_cluster.cc:1428] Setting key 3527824b71eb1c9de27a29bcf516b4a1
I20260806 22:27:06.818397 6219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:27:06 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:06.831549 6219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f: Bootstrap starting.
I20260806 22:27:06.835179 6219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:06.836301 6219 log.cc:824] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:06.839567 6219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f: No bootstrap required, opened a new log
I20260806 22:27:06.839680 6222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:06.817870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.1.8.126:36933)
0806 22:27:06.818719 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:06.818735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:06.818786 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:27:06.821194 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:06.821222 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:06.821737 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:06.822166 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:06.823496 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:06.825563 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:06.826795 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:06.827802 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:06.827991 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:06.832061 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:06.832072 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:06.832118 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:06.832235 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:06.835226 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:06.835988 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:06.836001 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:06.836007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:06.836251 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:06.836704 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:06.836710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:06.836713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:06.837554 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:06.837856 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:06.838269 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 22:27:06.838591 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":332,"threads_started":1}
I20260806 22:27:06.847786 6219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } }
I20260806 22:27:06.848186 6219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:06.848309 6219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230908391a0749b8a1e3c235bab9268f, State: Initialized, Role: FOLLOWER
I20260806 22:27:06.848912 6219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } }
I20260806 22:27:06.849176 6219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:06.849313 6219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:06.849529 6219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:06.851748 6219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } }
I20260806 22:27:06.852257 6219 leader_election.cc:302] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 230908391a0749b8a1e3c235bab9268f; no voters:
I20260806 22:27:06.852806 6219 leader_election.cc:288] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:06.852960 6226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:06.854256 6226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [term 1 LEADER]: Becoming Leader. State: Replica: 230908391a0749b8a1e3c235bab9268f, State: Running, Role: LEADER
I20260806 22:27:06.854774 6226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } }
I20260806 22:27:06.854979 6219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:06.857625 6230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 230908391a0749b8a1e3c235bab9268f. Latest consensus state: current_term: 1 leader_uuid: "230908391a0749b8a1e3c235bab9268f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } } }
I20260806 22:27:06.858044 6230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:06.858481 6227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "230908391a0749b8a1e3c235bab9268f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230908391a0749b8a1e3c235bab9268f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36933 } } }
I20260806 22:27:06.858836 6227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:06.867350 6244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:06.873050 6244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:06.881992 6244 catalog_manager.cc:1347] Generated new cluster ID: 2f1afe32eecb43b5946408c58df420d1
I20260806 22:27:06.882134 6244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:06.901510 6244 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:06.902803 6244 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:06.925009 6244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 230908391a0749b8a1e3c235bab9268f: Generated new TSK 0
I20260806 22:27:06.925871 6244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:06.989526 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:32807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:07.571635 6265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:07.572265 6265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:07.572552 6265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:07.585148 6265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:07.585417 6265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:07.585752 6265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:07.602877 6265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:32807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:07.605541 6265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:07.607950 6265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:07.633728 6270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:07.633935 6272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:07.634310 6277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:07.635939 6265 server_base.cc:1034] running on GCE node
I20260806 22:27:07.637077 6265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:07.638748 6265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:07.640007 6265 hybrid_clock.cc:648] HybridClock initialized: now 1786055227639929 us; error 69 us; skew 500 ppm
I20260806 22:27:07.640615 6265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:07 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:07.647132 6265 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:07.650341 6265 webserver.cc:460] Webserver started at http://127.1.8.65:43021/ using document root <none> and password file <none>
I20260806 22:27:07.652347 6265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:07.652604 6265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:07.653232 6265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:07.661140 6265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "fe3d5246171e4a3786f1e4ba494b3e3b"
format_stamp: "Formatted at 2026-08-06 22:27:07 on dist-test-slave-7vjv"
server_key: "7b0ccf0eb4252fdd9e1a442a1f31fbb1"
server_key_iv: "50cd081ef0aa6d996f891b4c0f534be0"
server_key_version: "encryptionkey@0"
I20260806 22:27:07.662717 6265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "fe3d5246171e4a3786f1e4ba494b3e3b"
format_stamp: "Formatted at 2026-08-06 22:27:07 on dist-test-slave-7vjv"
server_key: "7b0ccf0eb4252fdd9e1a442a1f31fbb1"
server_key_iv: "50cd081ef0aa6d996f891b4c0f534be0"
server_key_version: "encryptionkey@0"
I20260806 22:27:07.672374 6265 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.005s
I20260806 22:27:07.678520 6291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:07.681056 6265 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:27:07.681361 6265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "fe3d5246171e4a3786f1e4ba494b3e3b"
format_stamp: "Formatted at 2026-08-06 22:27:07 on dist-test-slave-7vjv"
server_key: "7b0ccf0eb4252fdd9e1a442a1f31fbb1"
server_key_iv: "50cd081ef0aa6d996f891b4c0f534be0"
server_key_version: "encryptionkey@0"
I20260806 22:27:07.681738 6265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:07.703581 6265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:07.711030 6265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:07.711588 6265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:07.713367 6265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:07.715819 6265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:07.716003 6265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:07.716295 6265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:07.716427 6265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:07.777000 6265 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:40875
I20260806 22:27:07.777066 6479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:40875 every 8 connection(s)
I20260806 22:27:07.779459 6265 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:07.781883 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 6265
I20260806 22:27:07.782282 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:07.783877 1057 external_mini_cluster.cc:1428] Setting key 5126e5249e0f05f7b4306e00351bd19b
Aug 06 22:27:07 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:07.854717 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:07.784664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:33155 (local address 127.1.8.126:36933)
0806 22:27:07.785651 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:07.785685 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:07.787629 (+ 1944us) server_negotiation.cc:407] Connection header received
0806 22:27:07.793357 (+ 5728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:07.793364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:07.793482 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:07.793635 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:07.798106 (+ 4471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:07.799731 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:07.804688 (+ 4957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:07.805413 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:07.805689 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:07.814672 (+ 8983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:07.814700 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:07.814707 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:07.814897 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:07.831052 (+ 16155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:07.843454 (+ 12402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:07.843509 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:07.843595 (+ 86us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:07.844511 (+ 916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:07.847246 (+ 2735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:07.847258 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:07.847262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:07.848530 (+ 1268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:07.848965 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:07.852406 (+ 3441us) server_negotiation.cc:299] Negotiation successful
0806 22:27:07.853242 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":507,"threads_started":1}
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
I20260806 22:27:07.862510 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:07.785872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36933 (local address 127.1.8.65:33155)
0806 22:27:07.787273 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:07.787396 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:07.792752 (+ 5356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:07.794187 (+ 1435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:07.794219 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:07.795865 (+ 1646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:07.797735 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:07.797791 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:07.800029 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:07.800041 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:07.804443 (+ 4402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:07.804475 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:07.805815 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:07.805834 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:07.806270 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:07.807638 (+ 1368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:07.807689 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:07.814105 (+ 6416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:07.833141 (+ 19036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:07.833327 (+ 186us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:07.833610 (+ 283us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:07.842808 (+ 9198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:07.844936 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:07.844943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:07.845008 (+ 65us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:07.846493 (+ 1485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:07.848946 (+ 2453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:07.848969 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:07.850373 (+ 1404us) client_negotiation.cc:769] Sending connection context
0806 22:27:07.851732 (+ 1359us) client_negotiation.cc:240] Negotiation successful
0806 22:27:07.854010 (+ 2278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":613,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
I20260806 22:27:07.877995 6480 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36933
I20260806 22:27:07.879622 6480 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:07.885802 6480 heartbeater.cc:511] Master 127.1.8.126:36933 requested a full tablet report, sending...
I20260806 22:27:07.897982 6163 ts_manager.cc:194] Registered new tserver with Master: fe3d5246171e4a3786f1e4ba494b3e3b (127.1.8.65:40875)
I20260806 22:27:07.902501 6163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:33155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
W20260806 22:27:07.941063 1057 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.8.66: real 0.152s user 0.002s sys 0.052s
I20260806 22:27:07.942022 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:32807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:08.871470 6489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:08.872205 6489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:08.872586 6489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:08.891654 6489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:08.891844 6489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:08.892040 6489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:08.908295 6480 heartbeater.cc:503] Master 127.1.8.126:36933 was elected leader, sending a full tablet report...
I20260806 22:27:08.925122 6489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:32807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:08.928375 6489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:08.931769 6489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:08.965190 6496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:08.965135 6497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:08.965294 6501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:08.968250 6489 server_base.cc:1034] running on GCE node
I20260806 22:27:08.969744 6489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:08.972034 6489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:08.975773 6489 hybrid_clock.cc:648] HybridClock initialized: now 1786055228975542 us; error 222 us; skew 500 ppm
I20260806 22:27:08.976864 6489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:08 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:08.984975 6489 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:08.988765 6489 webserver.cc:460] Webserver started at http://127.1.8.66:32869/ using document root <none> and password file <none>
I20260806 22:27:08.990506 6489 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:08.990880 6489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:08.991557 6489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:08.997448 6489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "3448e80b68114eb7a4bede7b0f76cd86"
format_stamp: "Formatted at 2026-08-06 22:27:08 on dist-test-slave-7vjv"
server_key: "f17afd47e73456bd5a1681f9b801063d"
server_key_iv: "407b9c0499ea76ab16e5746eab8ec5e1"
server_key_version: "encryptionkey@0"
I20260806 22:27:08.999054 6489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "3448e80b68114eb7a4bede7b0f76cd86"
format_stamp: "Formatted at 2026-08-06 22:27:08 on dist-test-slave-7vjv"
server_key: "f17afd47e73456bd5a1681f9b801063d"
server_key_iv: "407b9c0499ea76ab16e5746eab8ec5e1"
server_key_version: "encryptionkey@0"
I20260806 22:27:09.009413 6489 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260806 22:27:09.018077 6512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:09.021474 6489 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.000s
I20260806 22:27:09.021920 6489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "3448e80b68114eb7a4bede7b0f76cd86"
format_stamp: "Formatted at 2026-08-06 22:27:08 on dist-test-slave-7vjv"
server_key: "f17afd47e73456bd5a1681f9b801063d"
server_key_iv: "407b9c0499ea76ab16e5746eab8ec5e1"
server_key_version: "encryptionkey@0"
I20260806 22:27:09.022465 6489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:09.053959 6489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:09.076195 6489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:09.076712 6489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:09.078994 6489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:09.082011 6489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:09.082312 6489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:09.082574 6489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:09.082837 6489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:09.176143 6489 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:38457
I20260806 22:27:09.176232 6680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:38457 every 8 connection(s)
I20260806 22:27:09.179171 6489 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:09.183561 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 6489
I20260806 22:27:09.184075 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:09.185678 1057 external_mini_cluster.cc:1428] Setting key db50d76dcd1e7c97703cabd3922b2c17
Aug 06 22:27:09 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:09.230810 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:09.184787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:55115 (local address 127.1.8.126:36933)
0806 22:27:09.186288 (+ 1501us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:09.186306 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:09.189073 (+ 2767us) server_negotiation.cc:407] Connection header received
0806 22:27:09.193276 (+ 4203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:09.193293 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:09.193614 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:09.193825 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:09.197424 (+ 3599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:09.200230 (+ 2806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:09.203992 (+ 3762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:09.206174 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:09.206584 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:09.215851 (+ 9267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:09.215858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:09.215885 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:09.216038 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:09.222078 (+ 6040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:09.224543 (+ 2465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:09.224549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:09.224551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:09.224768 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:09.225733 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:09.225738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:09.225740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:09.226460 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:09.226730 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:09.229390 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0806 22:27:09.229876 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1092,"thread_start_us":688,"threads_started":1}
I20260806 22:27:09.235205 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:09.186615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36933 (local address 127.1.8.66:55115)
0806 22:27:09.188578 (+ 1963us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:09.188676 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:09.192674 (+ 3998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:09.194237 (+ 1563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:09.194253 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:09.195468 (+ 1215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:09.196988 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:09.197021 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:09.200511 (+ 3490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:09.200539 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:09.203546 (+ 3007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:09.203587 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:09.206747 (+ 3160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:09.206756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:09.207364 (+ 608us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:09.208892 (+ 1528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:09.209015 (+ 123us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:09.215314 (+ 6299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:09.222498 (+ 7184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:09.222520 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:09.222562 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:09.224250 (+ 1688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:09.224991 (+ 741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:09.225007 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:09.225015 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:09.225544 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:09.226809 (+ 1265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:09.226830 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:09.228174 (+ 1344us) client_negotiation.cc:769] Sending connection context
0806 22:27:09.229020 (+ 846us) client_negotiation.cc:240] Negotiation successful
0806 22:27:09.230712 (+ 1692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"thread_start_us":699,"threads_started":1}
I20260806 22:27:09.238901 6681 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36933
I20260806 22:27:09.239663 6681 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:09.241709 6681 heartbeater.cc:511] Master 127.1.8.126:36933 requested a full tablet report, sending...
I20260806 22:27:09.245067 6163 ts_manager.cc:194] Registered new tserver with Master: 3448e80b68114eb7a4bede7b0f76cd86 (127.1.8.66:38457)
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
I20260806 22:27:09.246497 6163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:55115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
W20260806 22:27:09.292073 1057 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.8.67: real 0.101s user 0.001s sys 0.039s
I20260806 22:27:09.293466 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:32807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:10.251350 6681 heartbeater.cc:503] Master 127.1.8.126:36933 was elected leader, sending a full tablet report...
W20260806 22:27:10.370911 6690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:10.371507 6690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:10.371788 6690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:10.384560 6690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:10.384845 6690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:10.385190 6690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:10.402608 6690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:32807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:10.405161 6690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:10.408018 6690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:10.437615 6705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:10.437922 6700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:10.437458 6701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:10.441417 6690 server_base.cc:1034] running on GCE node
I20260806 22:27:10.442667 6690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:10.444545 6690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:10.446048 6690 hybrid_clock.cc:648] HybridClock initialized: now 1786055230445921 us; error 123 us; skew 500 ppm
I20260806 22:27:10.446569 6690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:10 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:10.457022 6690 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:10.460323 6690 webserver.cc:460] Webserver started at http://127.1.8.67:46189/ using document root <none> and password file <none>
I20260806 22:27:10.461728 6690 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:10.461988 6690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:10.462613 6690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:10.468662 6690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "b32e2de3591a485e8afed596e70ef2a7"
format_stamp: "Formatted at 2026-08-06 22:27:10 on dist-test-slave-7vjv"
server_key: "4f547890a36d6ddcdf356c7366193842"
server_key_iv: "42ffcb0bd539d4dd3b0b9f911bc7fe3c"
server_key_version: "encryptionkey@0"
I20260806 22:27:10.470470 6690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "b32e2de3591a485e8afed596e70ef2a7"
format_stamp: "Formatted at 2026-08-06 22:27:10 on dist-test-slave-7vjv"
server_key: "4f547890a36d6ddcdf356c7366193842"
server_key_iv: "42ffcb0bd539d4dd3b0b9f911bc7fe3c"
server_key_version: "encryptionkey@0"
I20260806 22:27:10.480988 6690 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20260806 22:27:10.488572 6716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:10.491097 6690 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 22:27:10.491381 6690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "b32e2de3591a485e8afed596e70ef2a7"
format_stamp: "Formatted at 2026-08-06 22:27:10 on dist-test-slave-7vjv"
server_key: "4f547890a36d6ddcdf356c7366193842"
server_key_iv: "42ffcb0bd539d4dd3b0b9f911bc7fe3c"
server_key_version: "encryptionkey@0"
I20260806 22:27:10.491714 6690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:10.524201 6690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:10.531813 6690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:10.532482 6690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:10.534463 6690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:10.537403 6690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:10.537648 6690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:10.537952 6690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:10.538079 6690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:10.603673 6690 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:45777
I20260806 22:27:10.603722 6892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:45777 every 8 connection(s)
I20260806 22:27:10.606009 6690 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:10.614374 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 6690
I20260806 22:27:10.614702 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:10.615548 1057 external_mini_cluster.cc:1428] Setting key 657e52ba894747f6f51f46594c331268
Aug 06 22:27:10 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:10.644351 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:10.610230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:59965 (local address 127.1.8.126:36933)
0806 22:27:10.611121 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:10.611126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:10.612790 (+ 1664us) server_negotiation.cc:407] Connection header received
0806 22:27:10.616202 (+ 3412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:10.616220 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:10.616377 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:10.616644 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:10.620252 (+ 3608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.622454 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.625830 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.626688 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.626893 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:10.633076 (+ 6183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:10.633103 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:10.633111 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:10.633196 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:10.638344 (+ 5148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:10.640339 (+ 1995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:10.640346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:10.640352 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:10.640651 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:10.641722 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:10.641727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:10.641738 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:10.642214 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:10.642388 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:10.643515 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0806 22:27:10.643806 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":468,"threads_started":1}
I20260806 22:27:10.646131 6898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:10.611332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36933 (local address 127.1.8.67:59965)
0806 22:27:10.612517 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:10.612604 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:10.615635 (+ 3031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:10.616963 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:10.616991 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:10.618492 (+ 1501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:10.619919 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:10.619952 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.622735 (+ 2783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.622748 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:10.625690 (+ 2942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:10.625703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.626973 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.626987 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:10.627355 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:10.628754 (+ 1399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:10.628792 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:10.632698 (+ 3906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:10.638579 (+ 5881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:10.638621 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:10.638708 (+ 87us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:10.640055 (+ 1347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:10.640933 (+ 878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:10.640940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:10.640945 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:10.641524 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:10.642399 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:10.642407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:10.642974 (+ 567us) client_negotiation.cc:769] Sending connection context
0806 22:27:10.643391 (+ 417us) client_negotiation.cc:240] Negotiation successful
0806 22:27:10.644039 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":433,"threads_started":1}
I20260806 22:27:10.648865 6893 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36933
I20260806 22:27:10.649406 6893 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:10.650707 6893 heartbeater.cc:511] Master 127.1.8.126:36933 requested a full tablet report, sending...
I20260806 22:27:10.653188 6162 ts_manager.cc:194] Registered new tserver with Master: b32e2de3591a485e8afed596e70ef2a7 (127.1.8.67:45777)
I20260806 22:27:10.654732 6162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:59965
I20260806 22:27:10.662472 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:27:10 dist-test-slave-7vjv krb5kdc[6086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
W20260806 22:27:10.940026 1057 mini_kdc.cc:317] Time spent kinit for joe-interloper: real 0.275s user 0.003s sys 0.025s
Aug 06 22:27:10 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:10.982990 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:10.948192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.1.8.126:36933)
0806 22:27:10.948516 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:10.948524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:10.949339 (+ 815us) server_negotiation.cc:407] Connection header received
0806 22:27:10.949647 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:10.949655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:10.949834 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:10.950050 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:10.952730 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.954160 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.958269 (+ 4109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.959593 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.959725 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:10.969085 (+ 9360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:10.969103 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:10.969110 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:10.969200 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:10.976473 (+ 7273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:10.977884 (+ 1411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:10.977913 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:10.977917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:10.978346 (+ 429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:10.979076 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:10.979109 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:10.979112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:10.981245 (+ 2133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:10.981895 (+ 650us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:10.982232 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:27:10.982566 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260806 22:27:10.984413 6162 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41738
I20260806 22:27:10.987881 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6265
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:11.017795 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:10.990257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.1.8.65:40875)
0806 22:27:10.991517 (+ 1260us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:10.991534 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:10.992412 (+ 878us) server_negotiation.cc:407] Connection header received
0806 22:27:10.992645 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:10.992654 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:10.993025 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:10.993243 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:10.995288 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:10.997115 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:10.999023 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:11.000059 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:11.000329 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:11.007772 (+ 7443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:11.007802 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:11.007828 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:11.007884 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:11.012961 (+ 5077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.014092 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.014100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.014112 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.014325 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.014880 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.014883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.014885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.015690 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:11.016243 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:11.016728 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 22:27:11.017102 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":539,"threads_started":1}
W20260806 22:27:11.020061 6474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59660
W20260806 22:27:11.021981 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6265: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:27:11.022267 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 6265
I20260806 22:27:11.092721 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6489
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:27:11.118774 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:11.094159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.1.8.66:38457)
0806 22:27:11.095214 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:11.095233 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:11.095286 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:27:11.095396 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:11.095413 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:11.095800 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:11.096083 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:11.098116 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:11.099905 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:11.102312 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:11.103336 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:11.103506 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:11.110535 (+ 7029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:11.110558 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:11.110585 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:11.110654 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:11.115135 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.116034 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.116048 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.116050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.116346 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.116775 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.116778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.116781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.117325 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:11.117635 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:11.118039 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 22:27:11.118351 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":509,"threads_started":1}
W20260806 22:27:11.121117 6673 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54298
W20260806 22:27:11.122985 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6489: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:27:11.123234 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 6489
I20260806 22:27:11.173146 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6690
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055230, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:27:11.201407 6913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:11.174706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60392 (local address 127.1.8.67:45777)
0806 22:27:11.175692 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:11.175711 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:11.175757 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:27:11.175881 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:11.175897 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:11.176362 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:11.176592 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:11.178834 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:11.180706 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:11.182665 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:11.183649 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:11.183813 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:11.191364 (+ 7551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:11.191377 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:11.191406 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:11.191482 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:11.196802 (+ 5320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.197867 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.197875 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.197877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.198118 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:11.198655 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:11.198663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:11.198665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:11.199296 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:11.199696 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:11.200267 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 22:27:11.200708 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":594,"threads_started":1}
W20260806 22:27:11.203590 6885 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60392
W20260806 22:27:11.205497 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6690: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:27:11.205758 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 6690
I20260806 22:27:11.253111 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6105
I20260806 22:27:11.255153 6212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:11.426820 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 6105
2026-08-06T22:27:11Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (6689 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6917](info): set up 2 sockets
Aug 06 22:27:11 dist-test-slave-7vjv krb5kdc[6917](info): commencing operation
krb5kdc: starting...
W20260806 22:27:12.908393 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.008s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:13 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:13.135290 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45339
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:13.476639 6936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:13.476917 6936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:13.477005 6936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:13.484081 6936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:13.484187 6936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:13.484233 6936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:13.484290 6936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:13.484340 6936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:13.493786 6936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:13.494999 6936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:13.496361 6936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:13.508457 6942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:13.508423 6945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:13.508526 6943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:13.509632 6936 server_base.cc:1034] running on GCE node
I20260806 22:27:13.510399 6936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:13.511703 6936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:13.512907 6936 hybrid_clock.cc:648] HybridClock initialized: now 1786055233512861 us; error 43 us; skew 500 ppm
I20260806 22:27:13.513294 6936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:13 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055233, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:13.516366 6936 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:13.517877 6936 webserver.cc:460] Webserver started at http://127.1.8.126:39353/ using document root <none> and password file <none>
I20260806 22:27:13.518564 6936 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:13.518719 6936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:13.519048 6936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:13.522164 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "39691497c1f94b01950e6b5f8cc2866c"
format_stamp: "Formatted at 2026-08-06 22:27:13 on dist-test-slave-7vjv"
server_key: "515633e1c2eb774bf2aeaa72def0c862"
server_key_iv: "dbd2cb79881e558a9eb83de9d4f82a40"
server_key_version: "encryptionkey@0"
I20260806 22:27:13.523020 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "39691497c1f94b01950e6b5f8cc2866c"
format_stamp: "Formatted at 2026-08-06 22:27:13 on dist-test-slave-7vjv"
server_key: "515633e1c2eb774bf2aeaa72def0c862"
server_key_iv: "dbd2cb79881e558a9eb83de9d4f82a40"
server_key_version: "encryptionkey@0"
I20260806 22:27:13.528235 6936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 22:27:13.531551 6956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:13.532804 6936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 22:27:13.532991 6936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "39691497c1f94b01950e6b5f8cc2866c"
format_stamp: "Formatted at 2026-08-06 22:27:13 on dist-test-slave-7vjv"
server_key: "515633e1c2eb774bf2aeaa72def0c862"
server_key_iv: "dbd2cb79881e558a9eb83de9d4f82a40"
server_key_version: "encryptionkey@0"
I20260806 22:27:13.533196 6936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:13.543488 6936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:13.547231 6936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:13.547508 6936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:13.565984 6936 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:45339
I20260806 22:27:13.566071 7036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:45339 every 8 connection(s)
I20260806 22:27:13.567360 6936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:13.571779 7037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:13.576649 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 6936
I20260806 22:27:13.576895 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:13.577735 1057 external_mini_cluster.cc:1428] Setting key 7b7c19cbe8c15d61d8848058f4dae248
I20260806 22:27:13.587023 7037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c: Bootstrap starting.
I20260806 22:27:13.591334 7037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:13.592679 7037 log.cc:824] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:27:13 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:13.595453 7037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c: No bootstrap required, opened a new log
I20260806 22:27:13.601243 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:13.579840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.1.8.126:45339)
0806 22:27:13.580645 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:13.580656 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:13.580697 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:27:13.582276 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:13.582298 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:13.582689 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:13.583000 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:13.585013 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:13.586291 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:13.588147 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:13.588815 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:13.588938 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:13.593787 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:13.593795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:13.593809 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:13.593844 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:13.597008 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:13.597758 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:13.597764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:13.597770 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:13.597961 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:13.598359 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:13.598374 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:13.598377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:13.598905 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:13.599240 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:13.599815 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 22:27:13.600262 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":394,"threads_started":1}
I20260806 22:27:13.603694 7037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } }
I20260806 22:27:13.604169 7037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:13.604272 7037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39691497c1f94b01950e6b5f8cc2866c, State: Initialized, Role: FOLLOWER
I20260806 22:27:13.604848 7037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } }
I20260806 22:27:13.605073 7037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:13.605170 7037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:13.605343 7037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:13.607383 7037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } }
I20260806 22:27:13.607807 7037 leader_election.cc:302] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39691497c1f94b01950e6b5f8cc2866c; no voters:
I20260806 22:27:13.608421 7037 leader_election.cc:288] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:13.608520 7043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:13.609323 7043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [term 1 LEADER]: Becoming Leader. State: Replica: 39691497c1f94b01950e6b5f8cc2866c, State: Running, Role: LEADER
I20260806 22:27:13.610001 7043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } }
I20260806 22:27:13.610845 7037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:13.613668 7044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39691497c1f94b01950e6b5f8cc2866c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } } }
I20260806 22:27:13.614817 7044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:13.614447 7045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39691497c1f94b01950e6b5f8cc2866c. Latest consensus state: current_term: 1 leader_uuid: "39691497c1f94b01950e6b5f8cc2866c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39691497c1f94b01950e6b5f8cc2866c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45339 } } }
I20260806 22:27:13.615350 7045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:13.621598 7068 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:27:13.621937 7061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:13.627177 7061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:13.635926 7061 catalog_manager.cc:1347] Generated new cluster ID: 1625203b8f1a4502ba4b2f7a76234f06
I20260806 22:27:13.636055 7061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:13.687490 7061 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:13.689427 7061 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:13.698014 7061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39691497c1f94b01950e6b5f8cc2866c: Generated new TSK 0
I20260806 22:27:13.699064 7061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:13.763489 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:14.105381 7073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:14.105718 7073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:14.105849 7073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:14.113811 7073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:14.113963 7073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:14.114147 7073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:14.124099 7073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:14.125393 7073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:14.126689 7073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:14.139815 7080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:14.139901 7086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:14.139940 7082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:14.141376 7073 server_base.cc:1034] running on GCE node
I20260806 22:27:14.142009 7073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:14.142932 7073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:14.144181 7073 hybrid_clock.cc:648] HybridClock initialized: now 1786055234144112 us; error 46 us; skew 500 ppm
I20260806 22:27:14.144497 7073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:14 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:14.148182 7073 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:14.149590 7073 webserver.cc:460] Webserver started at http://127.1.8.65:35989/ using document root <none> and password file <none>
I20260806 22:27:14.150213 7073 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:14.150314 7073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:14.150578 7073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:14.153640 7073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "6e46ff3b09ee47ebb2485d9d0aa5bae3"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4bc72fcad6cf83821105d43f544c0a33"
server_key_iv: "c80ab5c8315efb04e143e51e63669fd8"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.154364 7073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "6e46ff3b09ee47ebb2485d9d0aa5bae3"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4bc72fcad6cf83821105d43f544c0a33"
server_key_iv: "c80ab5c8315efb04e143e51e63669fd8"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.159564 7073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 22:27:14.162974 7105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.164386 7073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:14.164568 7073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "6e46ff3b09ee47ebb2485d9d0aa5bae3"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4bc72fcad6cf83821105d43f544c0a33"
server_key_iv: "c80ab5c8315efb04e143e51e63669fd8"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.164772 7073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:14.195246 7073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:14.199517 7073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:14.199827 7073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:14.200935 7073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:14.202289 7073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:14.202412 7073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.202591 7073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:14.202677 7073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.236503 7073 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:36161
I20260806 22:27:14.236568 7280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:36161 every 8 connection(s)
I20260806 22:27:14.238108 7073 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:14.248500 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 7073
I20260806 22:27:14.248720 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:14.249642 1057 external_mini_cluster.cc:1428] Setting key 61ed05e0fce5a9a83b2ffe157e662019
Aug 06 22:27:14 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:14.258807 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:14.240681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:39505 (local address 127.1.8.126:45339)
0806 22:27:14.241111 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:14.241115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:14.242114 (+ 999us) server_negotiation.cc:407] Connection header received
0806 22:27:14.243990 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:14.243995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:14.244140 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:14.244269 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:14.245783 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.246512 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.248154 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.248655 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.248798 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:14.253047 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:14.253057 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:14.253061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:14.253092 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:14.255729 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:14.256557 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:14.256563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:14.256569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:14.256692 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:14.257124 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:14.257131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:14.257134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:14.257398 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:14.257570 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:14.258232 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 22:27:14.258455 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":208,"threads_started":1}
I20260806 22:27:14.259822 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:14.241254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45339 (local address 127.1.8.65:39505)
0806 22:27:14.241942 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:14.241980 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:14.243712 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:14.244520 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:14.244528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:14.245044 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:14.245606 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:14.245624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.246692 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.246696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:14.247908 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:14.247921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.248905 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.248913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:14.249224 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:14.250020 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:14.250043 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:14.252795 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:14.255870 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:14.255878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:14.255895 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:14.256418 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:14.256813 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:14.256817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:14.256821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:14.257013 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:14.257546 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:14.257554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:14.257873 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 22:27:14.258136 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 22:27:14.258678 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":290,"threads_started":1}
I20260806 22:27:14.261426 7281 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45339
I20260806 22:27:14.261778 7281 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:14.262738 7281 heartbeater.cc:511] Master 127.1.8.126:45339 requested a full tablet report, sending...
I20260806 22:27:14.265311 6984 ts_manager.cc:194] Registered new tserver with Master: 6e46ff3b09ee47ebb2485d9d0aa5bae3 (127.1.8.65:36161)
I20260806 22:27:14.267607 6984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:39505
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:27:14.306952 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:14.648806 7290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:14.649070 7290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:14.649163 7290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:14.656358 7290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:14.656467 7290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:14.656579 7290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:14.666435 7290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:14.667660 7290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:14.669317 7290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:14.682513 7298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:14.682477 7299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:14.683013 7305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:14.683255 7290 server_base.cc:1034] running on GCE node
I20260806 22:27:14.683851 7290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:14.684819 7290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:14.686054 7290 hybrid_clock.cc:648] HybridClock initialized: now 1786055234686010 us; error 38 us; skew 500 ppm
I20260806 22:27:14.686374 7290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:14 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:14.689555 7290 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:14.691001 7290 webserver.cc:460] Webserver started at http://127.1.8.66:35335/ using document root <none> and password file <none>
I20260806 22:27:14.691715 7290 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:14.691864 7290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:14.692260 7290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:14.695186 7290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "6ba4665fd283484c8817af0f201cb4fa"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4d7cc8f5bbea03e02ab66b2b7189ae52"
server_key_iv: "3d07bbab22798ce36d3f1c1db62b9508"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.695988 7290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "6ba4665fd283484c8817af0f201cb4fa"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4d7cc8f5bbea03e02ab66b2b7189ae52"
server_key_iv: "3d07bbab22798ce36d3f1c1db62b9508"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.701082 7290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:27:14.704489 7317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.705789 7290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:27:14.705963 7290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "6ba4665fd283484c8817af0f201cb4fa"
format_stamp: "Formatted at 2026-08-06 22:27:14 on dist-test-slave-7vjv"
server_key: "4d7cc8f5bbea03e02ab66b2b7189ae52"
server_key_iv: "3d07bbab22798ce36d3f1c1db62b9508"
server_key_version: "encryptionkey@0"
I20260806 22:27:14.706166 7290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:14.721511 7290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:14.725389 7290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:14.725674 7290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:14.726651 7290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:14.727953 7290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:14.728058 7290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.728231 7290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:14.728312 7290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:14.762813 7290 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:34875
I20260806 22:27:14.762863 7493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:34875 every 8 connection(s)
I20260806 22:27:14.764248 7290 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:14.770457 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 7290
I20260806 22:27:14.770660 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:14.771229 1057 external_mini_cluster.cc:1428] Setting key 6756e2df91c029ca009c41015ba38478
Aug 06 22:27:14 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:14.787331 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:14.766872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:47979 (local address 127.1.8.126:45339)
0806 22:27:14.767327 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:14.767331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:14.768430 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 22:27:14.770765 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:14.770773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:14.770861 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:14.770951 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:14.772971 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.774052 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.776677 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.777128 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.777226 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:14.781491 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:14.781496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:14.781500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:14.781532 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:14.784177 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:14.784904 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:14.784909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:14.784919 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:14.785047 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:14.785486 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:14.785490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:14.785492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:14.785750 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:14.785920 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:14.786591 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 22:27:14.786815 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
I20260806 22:27:14.788378 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:14.767430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45339 (local address 127.1.8.66:47979)
0806 22:27:14.768241 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:14.768290 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:14.770399 (+ 2109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:14.771255 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:14.771269 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:14.772004 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:14.772779 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:14.772797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.774245 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.774252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:14.776516 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:14.776534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:14.777361 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:14.777371 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:14.777711 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:14.778460 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:14.778495 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:14.781248 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:14.784308 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:14.784317 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:14.784335 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:14.784768 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:14.785163 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:14.785168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:14.785171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:14.785358 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:14.785880 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:14.785891 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:14.786228 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 22:27:14.786491 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 22:27:14.787151 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":277,"threads_started":1}
I20260806 22:27:14.789897 7496 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45339
I20260806 22:27:14.790201 7496 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:14.790957 7496 heartbeater.cc:511] Master 127.1.8.126:45339 requested a full tablet report, sending...
I20260806 22:27:14.792562 6984 ts_manager.cc:194] Registered new tserver with Master: 6ba4665fd283484c8817af0f201cb4fa (127.1.8.66:34875)
I20260806 22:27:14.793907 6984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:47979
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:27:14.827358 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:15.162842 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:15.163126 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:15.163287 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:15.170586 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:15.170696 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:15.170912 7509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:15.182045 7509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:15.183375 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:15.184746 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:15.197899 7520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:15.197852 7521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:15.197923 7526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:15.198817 7509 server_base.cc:1034] running on GCE node
I20260806 22:27:15.199359 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:15.200322 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:15.201514 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786055235201462 us; error 51 us; skew 500 ppm
I20260806 22:27:15.201773 7509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:15.204989 7509 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:15.206434 7509 webserver.cc:460] Webserver started at http://127.1.8.67:43193/ using document root <none> and password file <none>
I20260806 22:27:15.207108 7509 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:15.207257 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:15.207568 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:15.210764 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "6d091722da8f42b8b2c26c6f59a5d20d"
format_stamp: "Formatted at 2026-08-06 22:27:15 on dist-test-slave-7vjv"
server_key: "4221e2f607a0251cc00d21420eef2022"
server_key_iv: "66f4e76e2fce35cbb6e3864aede158c8"
server_key_version: "encryptionkey@0"
I20260806 22:27:15.211735 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "6d091722da8f42b8b2c26c6f59a5d20d"
format_stamp: "Formatted at 2026-08-06 22:27:15 on dist-test-slave-7vjv"
server_key: "4221e2f607a0251cc00d21420eef2022"
server_key_iv: "66f4e76e2fce35cbb6e3864aede158c8"
server_key_version: "encryptionkey@0"
I20260806 22:27:15.217113 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 22:27:15.220557 7536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:15.221870 7509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:27:15.222069 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "6d091722da8f42b8b2c26c6f59a5d20d"
format_stamp: "Formatted at 2026-08-06 22:27:15 on dist-test-slave-7vjv"
server_key: "4221e2f607a0251cc00d21420eef2022"
server_key_iv: "66f4e76e2fce35cbb6e3864aede158c8"
server_key_version: "encryptionkey@0"
I20260806 22:27:15.222291 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:15.237633 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:15.242365 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:15.242681 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:15.243656 7509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:15.245014 7509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:15.245119 7509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:15.245273 7509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:15.245353 7509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:15.271400 7281 heartbeater.cc:503] Master 127.1.8.126:45339 was elected leader, sending a full tablet report...
I20260806 22:27:15.279774 7509 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:39929
I20260806 22:27:15.279794 7702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:39929 every 8 connection(s)
I20260806 22:27:15.281224 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:15.290267 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 7509
I20260806 22:27:15.290449 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:15.290949 1057 external_mini_cluster.cc:1428] Setting key 680bc8dc2d8a0f36ea270b6824c50a08
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:15.301342 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.283938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:51879 (local address 127.1.8.126:45339)
0806 22:27:15.284190 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.284194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.285345 (+ 1151us) server_negotiation.cc:407] Connection header received
0806 22:27:15.287046 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.287050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.287116 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.287202 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.288712 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.289433 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.291135 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.291528 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.291687 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.295565 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.295570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.295573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.295611 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.298304 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.299021 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.299030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.299032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.299148 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.299637 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.299642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.299644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.299914 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.300073 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.300783 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.301029 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:27:15.302289 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.284457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45339 (local address 127.1.8.67:51879)
0806 22:27:15.285188 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:15.285226 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:15.286799 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:15.287418 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:15.287426 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:15.287934 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:15.288566 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:15.288583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.289577 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.289580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:15.290949 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:15.290962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.291775 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.291782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:15.292128 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.292862 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:15.292884 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:15.295355 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:15.298456 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:15.298465 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:15.298480 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:15.298893 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:15.299276 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:15.299281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:15.299285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:15.299487 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:15.300048 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:15.300056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:15.300405 (+ 349us) client_negotiation.cc:769] Sending connection context
0806 22:27:15.300662 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 22:27:15.301188 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":278,"threads_started":1}
I20260806 22:27:15.303615 7703 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45339
I20260806 22:27:15.303890 7703 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:15.304677 7703 heartbeater.cc:511] Master 127.1.8.126:45339 requested a full tablet report, sending...
I20260806 22:27:15.306293 6983 ts_manager.cc:194] Registered new tserver with Master: 6d091722da8f42b8b2c26c6f59a5d20d (127.1.8.67:39929)
I20260806 22:27:15.307153 6983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:51879
I20260806 22:27:15.318250 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:15.375630 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.359793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.1.8.65:36161)
0806 22:27:15.360555 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.360563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.360601 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:27:15.360663 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.360669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.360878 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.361025 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.362369 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.363373 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.364823 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.365385 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.365485 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.369917 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.369928 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.369943 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.369979 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.372974 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.373633 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.373639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.373642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.373762 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.374154 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.374157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.374159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.374484 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.374717 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.374974 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.375181 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":462,"threads_started":1}
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:15.393815 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.380090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.1.8.126:45339)
0806 22:27:15.380321 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.380327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.380550 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:27:15.380758 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.380762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.380824 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.380927 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.382140 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.382879 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.384178 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.384584 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.384721 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.388550 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.388554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.388558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.388594 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.391110 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.391791 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.391801 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.391803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.391930 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.392347 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.392352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.392357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.392676 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.392947 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.393240 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.393465 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 22:27:15.394378 6983 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996
I20260806 22:27:15.395514 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 7073
I20260806 22:27:15.407353 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.396402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.1.8.65:36161)
0806 22:27:15.396621 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.396627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.396866 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:27:15.397028 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.397031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.397119 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.397195 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.398255 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.398890 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.400077 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.400515 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.400615 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.402595 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.402599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.402602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.402632 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.405225 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.405836 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.405839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.405841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.405958 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.406255 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.406258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.406260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.406486 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.406727 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.406851 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.407008 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:27:15.407884 7275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:15.492405 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 7073
I20260806 22:27:15.512449 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 7290
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:27:15.532773 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.513339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.1.8.66:34875)
0806 22:27:15.513878 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.513886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.513910 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:27:15.513979 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.513985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.514218 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.514399 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.515808 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.517070 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.518339 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.518927 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.519071 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.526670 (+ 7599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.526683 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.526699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.526755 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.529802 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.530505 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.530514 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.530517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.530652 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.531059 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.531065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.531068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.531437 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.531763 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.532127 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.532412 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":271,"threads_started":1}
I20260806 22:27:15.534176 7488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:15.623649 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 7290
I20260806 22:27:15.648233 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 7509
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[6917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:27:15.663821 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:15.649159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.1.8.67:39929)
0806 22:27:15.649817 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:15.649825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:15.649856 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:27:15.649921 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:15.649927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:15.650144 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:15.650275 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:15.651509 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.652593 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.653799 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:15.654333 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:15.654426 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:15.658845 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:15.658857 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:15.658870 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:15.658903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:15.661532 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.662123 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.662129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.662131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.662242 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:15.662548 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:15.662551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:15.662553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:15.662877 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:15.663113 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:15.663331 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:27:15.663540 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":318,"threads_started":1}
I20260806 22:27:15.665176 7695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:15.746696 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 7509
I20260806 22:27:15.772691 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 6936
I20260806 22:27:15.773878 7032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:15.880030 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 6936
2026-08-06T22:27:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4392 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[7731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[7731](info): set up 2 sockets
Aug 06 22:27:15 dist-test-slave-7vjv krb5kdc[7731](info): commencing operation
krb5kdc: starting...
W20260806 22:27:17.229372 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.008s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:17 dist-test-slave-7vjv krb5kdc[7731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:17.465682 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45551
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:17.799216 7751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:17.799495 7751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:17.799587 7751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:17.806859 7751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:17.806964 7751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:17.807008 7751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:17.807056 7751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:17.807103 7751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:17.816749 7751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:17.817955 7751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:17.819209 7751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:17.831096 7762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:17.831049 7758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:17.831130 7757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:17.831984 7751 server_base.cc:1034] running on GCE node
I20260806 22:27:17.832821 7751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:17.834098 7751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:17.835335 7751 hybrid_clock.cc:648] HybridClock initialized: now 1786055237835307 us; error 58 us; skew 500 ppm
I20260806 22:27:17.835714 7751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:17 dist-test-slave-7vjv krb5kdc[7731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:17.838938 7751 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:17.840207 7751 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:17.840301 7751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:17.840576 7751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:17.844065 7751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "fda50f435d7a4b6f9b459a6442a0475a"
format_stamp: "Formatted at 2026-08-06 22:27:17 on dist-test-slave-7vjv"
server_key: "845813ac27a77d7dc84f572e2ecea6c1"
server_key_iv: "22be0614ae6f045326ada896b2644ed1"
server_key_version: "encryptionkey@0"
I20260806 22:27:17.844961 7751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "fda50f435d7a4b6f9b459a6442a0475a"
format_stamp: "Formatted at 2026-08-06 22:27:17 on dist-test-slave-7vjv"
server_key: "845813ac27a77d7dc84f572e2ecea6c1"
server_key_iv: "22be0614ae6f045326ada896b2644ed1"
server_key_version: "encryptionkey@0"
I20260806 22:27:17.850016 7751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:17.853438 7774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:17.854709 7751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:17.854851 7751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "fda50f435d7a4b6f9b459a6442a0475a"
format_stamp: "Formatted at 2026-08-06 22:27:17 on dist-test-slave-7vjv"
server_key: "845813ac27a77d7dc84f572e2ecea6c1"
server_key_iv: "22be0614ae6f045326ada896b2644ed1"
server_key_version: "encryptionkey@0"
I20260806 22:27:17.855038 7751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:17.878168 7751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:17.882665 7751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:17.882892 7751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:17.906454 7751 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:45551
I20260806 22:27:17.906491 7852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:45551 every 8 connection(s)
I20260806 22:27:17.908130 7751 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:17.912904 7853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:17.918480 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 7751
I20260806 22:27:17.918700 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:17.920125 1057 external_mini_cluster.cc:1428] Setting key ae7239860d8d5757e2657d0404e48ceb
I20260806 22:27:17.934406 7853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: Bootstrap starting.
I20260806 22:27:17.941198 7853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:17.944041 7853 log.cc:824] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:27:17 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:17.949729 7853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: No bootstrap required, opened a new log
I20260806 22:27:17.965099 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:17.923477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.1.8.126:45551)
0806 22:27:17.924600 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:17.924612 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:17.924674 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:27:17.926747 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:17.926769 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:17.927185 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:17.927649 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:17.931416 (+ 3767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:17.933979 (+ 2563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:17.936269 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:17.937510 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:17.937685 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:17.946742 (+ 9057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:17.946765 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:17.946793 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:17.946860 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:17.952711 (+ 5851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:17.956181 (+ 3470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:17.956245 (+ 64us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:17.956259 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:17.956782 (+ 523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:17.957604 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:17.957608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:17.957610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:17.958738 (+ 1128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:17.959254 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:17.960335 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0806 22:27:17.961662 (+ 1327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":545,"threads_started":1}
I20260806 22:27:17.972822 7853 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } }
I20260806 22:27:17.973905 7853 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:17.974161 7853 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda50f435d7a4b6f9b459a6442a0475a, State: Initialized, Role: FOLLOWER
I20260806 22:27:17.975948 7853 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } }
I20260806 22:27:17.976712 7853 raft_consensus.cc:388] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:17.977284 7853 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:17.977979 7853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:17.983709 7853 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } }
I20260806 22:27:17.985196 7853 leader_election.cc:302] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fda50f435d7a4b6f9b459a6442a0475a; no voters:
I20260806 22:27:17.986770 7853 leader_election.cc:288] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:17.987113 7860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:17.989416 7860 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [term 1 LEADER]: Becoming Leader. State: Replica: fda50f435d7a4b6f9b459a6442a0475a, State: Running, Role: LEADER
I20260806 22:27:17.990602 7860 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } }
I20260806 22:27:17.991824 7853 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:17.997562 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fda50f435d7a4b6f9b459a6442a0475a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } } }
I20260806 22:27:17.997689 7863 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda50f435d7a4b6f9b459a6442a0475a. Latest consensus state: current_term: 1 leader_uuid: "fda50f435d7a4b6f9b459a6442a0475a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda50f435d7a4b6f9b459a6442a0475a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45551 } } }
I20260806 22:27:17.998415 7863 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:17.998417 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:18.000857 7871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:18.009541 7871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:18.025288 7871 catalog_manager.cc:1347] Generated new cluster ID: d6cc0d4ba4bc45ec8ce6a084029b2aa6
I20260806 22:27:18.025424 7871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:18.095258 7871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:18.098279 7871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:18.108001 7871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: Generated new TSK 0
I20260806 22:27:18.109512 7871 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:18.218708 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:19.070125 7894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:19.070940 7894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:19.071303 7894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:19.090754 7894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:19.091162 7894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:19.091467 7894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:19.124178 7894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:19.127488 7894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:19.131721 7894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:19.161156 7900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:19.161191 7905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:19.163395 7901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:19.165429 7894 server_base.cc:1034] running on GCE node
I20260806 22:27:19.166116 7894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:19.167375 7894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:19.168625 7894 hybrid_clock.cc:648] HybridClock initialized: now 1786055239168567 us; error 50 us; skew 500 ppm
I20260806 22:27:19.168998 7894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:19 dist-test-slave-7vjv krb5kdc[7731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:19.172667 7894 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:19.173979 7894 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:19.174118 7894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:19.174466 7894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:19.177752 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "7b23a8ff67be449eacbd751056ee9660"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "8fbae1b5848b9898891cff3b08cf9d85"
server_key_iv: "e4688550052b929c3f229c83c913ac2d"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.178704 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "7b23a8ff67be449eacbd751056ee9660"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "8fbae1b5848b9898891cff3b08cf9d85"
server_key_iv: "e4688550052b929c3f229c83c913ac2d"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.184834 7894 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 22:27:19.188573 7913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.190119 7894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:27:19.190356 7894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "7b23a8ff67be449eacbd751056ee9660"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "8fbae1b5848b9898891cff3b08cf9d85"
server_key_iv: "e4688550052b929c3f229c83c913ac2d"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.190714 7894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:19.207844 7894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:19.213229 7894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:19.213558 7894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:19.214679 7894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:19.216321 7894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:19.216460 7894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.216610 7894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:19.216751 7894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.262717 7894 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:37027
I20260806 22:27:19.262842 8093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:37027 every 8 connection(s)
I20260806 22:27:19.263754 7894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:19.264912 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 7894
I20260806 22:27:19.265208 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:19.266090 1057 external_mini_cluster.cc:1428] Setting key a590cb9faea1b2b2a336d51122e5b7af
Aug 06 22:27:19 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:19.286969 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:19.267178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:51787 (local address 127.1.8.126:45551)
0806 22:27:19.267763 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:19.267768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:19.268686 (+ 918us) server_negotiation.cc:407] Connection header received
0806 22:27:19.270842 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:19.270848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:19.270957 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:19.271153 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:19.272860 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.273643 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.275234 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.275668 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.275805 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:19.279475 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:19.279484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:19.279492 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:19.279536 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:19.282867 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:19.283951 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:19.283955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:19.283958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:19.284221 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:19.284852 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:19.284855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:19.284857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:19.285202 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:19.285327 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:19.286411 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0806 22:27:19.286632 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":288,"threads_started":1}
I20260806 22:27:19.288836 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:19.267474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45551 (local address 127.1.8.65:51787)
0806 22:27:19.268470 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:19.268576 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:19.270509 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:19.271514 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:19.271526 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:19.272050 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:19.272689 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:19.272705 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.273791 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.273797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:19.275082 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:19.275094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.275906 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.275915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:19.276246 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:19.276959 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:19.276986 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:19.279178 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:19.283054 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:19.283065 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:19.283084 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:19.283822 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:19.284361 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:19.284369 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:19.284375 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:19.284671 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:19.285336 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:19.285352 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:19.285965 (+ 613us) client_negotiation.cc:769] Sending connection context
0806 22:27:19.286319 (+ 354us) client_negotiation.cc:240] Negotiation successful
0806 22:27:19.287006 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":388,"threads_started":1}
I20260806 22:27:19.290730 8094 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45551
I20260806 22:27:19.291066 8094 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:19.291858 8094 heartbeater.cc:511] Master 127.1.8.126:45551 requested a full tablet report, sending...
I20260806 22:27:19.294559 7802 ts_manager.cc:194] Registered new tserver with Master: 7b23a8ff67be449eacbd751056ee9660 (127.1.8.65:37027)
I20260806 22:27:19.296478 7802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:51787
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:27:19.331583 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:19.718848 8105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:19.719177 8105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:19.719372 8105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:19.726840 8105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:19.727037 8105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:19.727246 8105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:19.737252 8105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:19.738595 8105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:19.739981 8105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:19.754282 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:19.754384 8117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:19.754427 8113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:19.755805 8105 server_base.cc:1034] running on GCE node
I20260806 22:27:19.756429 8105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:19.757352 8105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:19.758549 8105 hybrid_clock.cc:648] HybridClock initialized: now 1786055239758506 us; error 35 us; skew 500 ppm
I20260806 22:27:19.758950 8105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:19 dist-test-slave-7vjv krb5kdc[7731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:19.762327 8105 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:19.763514 8105 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:19.763670 8105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:19.764020 8105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:19.767095 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "e90f4f6b65d4457bb4607ee52fe1ef45"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "6f34d9fb28d2c9e6a2acffdd7ab64a1f"
server_key_iv: "a45656c6d0cee8247e7e2a57f2cb7c34"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.768040 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "e90f4f6b65d4457bb4607ee52fe1ef45"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "6f34d9fb28d2c9e6a2acffdd7ab64a1f"
server_key_iv: "a45656c6d0cee8247e7e2a57f2cb7c34"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.774214 8105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:27:19.778501 8123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.779917 8105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:27:19.780193 8105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "e90f4f6b65d4457bb4607ee52fe1ef45"
format_stamp: "Formatted at 2026-08-06 22:27:19 on dist-test-slave-7vjv"
server_key: "6f34d9fb28d2c9e6a2acffdd7ab64a1f"
server_key_iv: "a45656c6d0cee8247e7e2a57f2cb7c34"
server_key_version: "encryptionkey@0"
I20260806 22:27:19.780431 8105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:19.799571 8105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:19.803834 8105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:19.804203 8105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:19.805209 8105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:19.806658 8105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:19.806808 8105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.806937 8105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:19.807034 8105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:19.844837 8105 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:46097
I20260806 22:27:19.844939 8311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:46097 every 8 connection(s)
I20260806 22:27:19.845781 8105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:19.846997 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 8105
I20260806 22:27:19.847188 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:19.847818 1057 external_mini_cluster.cc:1428] Setting key 451ef3d102f8e3cc8886d5f7509c6035
Aug 06 22:27:19 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:19.871390 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:19.848707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:57479 (local address 127.1.8.126:45551)
0806 22:27:19.849185 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:19.849190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:19.850363 (+ 1173us) server_negotiation.cc:407] Connection header received
0806 22:27:19.853378 (+ 3015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:19.853382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:19.853457 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:19.853549 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:19.855751 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.856616 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.859141 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.859643 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.859741 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:19.863706 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:19.863715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:19.863723 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:19.863768 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:19.867808 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:19.868677 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:19.868685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:19.868691 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:19.868861 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:19.869290 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:19.869297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:19.869303 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:19.869767 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:19.869955 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:19.870658 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 22:27:19.870928 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260806 22:27:19.872082 8315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:19.849256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45551 (local address 127.1.8.66:57479)
0806 22:27:19.850186 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:19.850246 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:19.853092 (+ 2846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:19.853840 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:19.853855 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:19.854701 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:19.855557 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:19.855599 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.856776 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.856784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:19.859004 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:19.859023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:19.859885 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:19.859896 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:19.860236 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:19.861003 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:19.861037 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:19.863467 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:19.867989 (+ 4522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:19.867998 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:19.868016 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:19.868536 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:19.868973 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:19.868976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:19.868978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:19.869166 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:19.869899 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:19.869905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:19.870242 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 22:27:19.870473 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 22:27:19.870913 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260806 22:27:19.873703 8312 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45551
I20260806 22:27:19.874068 8312 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:19.874994 8312 heartbeater.cc:511] Master 127.1.8.126:45551 requested a full tablet report, sending...
I20260806 22:27:19.876775 7802 ts_manager.cc:194] Registered new tserver with Master: e90f4f6b65d4457bb4607ee52fe1ef45 (127.1.8.66:46097)
I20260806 22:27:19.877620 7802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:57479
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:27:19.909916 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:20.299827 8094 heartbeater.cc:503] Master 127.1.8.126:45551 was elected leader, sending a full tablet report...
W20260806 22:27:20.317273 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:20.317734 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:20.317950 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:20.325977 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:20.326136 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:20.326327 8319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:20.341576 8319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:20.343633 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:20.345291 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:20.360718 8330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:20.360693 8334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:20.360630 8336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:20.361927 8319 server_base.cc:1034] running on GCE node
I20260806 22:27:20.362498 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:20.363457 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:20.364698 8319 hybrid_clock.cc:648] HybridClock initialized: now 1786055240364644 us; error 58 us; skew 500 ppm
I20260806 22:27:20.365036 8319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:20 dist-test-slave-7vjv krb5kdc[7731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055240, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:20.368606 8319 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:20.369951 8319 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:20.370112 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:20.370641 8319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:20.374502 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "81999fd97b2a4279b7e6b023629b8384"
format_stamp: "Formatted at 2026-08-06 22:27:20 on dist-test-slave-7vjv"
server_key: "e26fc051dd064e54023beb0fa04efc25"
server_key_iv: "ab91d47e18186fe44675caa31a45d179"
server_key_version: "encryptionkey@0"
I20260806 22:27:20.375626 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "81999fd97b2a4279b7e6b023629b8384"
format_stamp: "Formatted at 2026-08-06 22:27:20 on dist-test-slave-7vjv"
server_key: "e26fc051dd064e54023beb0fa04efc25"
server_key_iv: "ab91d47e18186fe44675caa31a45d179"
server_key_version: "encryptionkey@0"
I20260806 22:27:20.382835 8319 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:27:20.387226 8346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:20.389525 8319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:27:20.389874 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "81999fd97b2a4279b7e6b023629b8384"
format_stamp: "Formatted at 2026-08-06 22:27:20 on dist-test-slave-7vjv"
server_key: "e26fc051dd064e54023beb0fa04efc25"
server_key_iv: "ab91d47e18186fe44675caa31a45d179"
server_key_version: "encryptionkey@0"
I20260806 22:27:20.390300 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:20.413945 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:20.418473 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:20.418802 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:20.420310 8319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:20.421870 8319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:20.422022 8319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:20.422166 8319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:20.422266 8319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:20.466090 8319 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:43751
I20260806 22:27:20.466151 8514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:43751 every 8 connection(s)
I20260806 22:27:20.467147 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:20.469193 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 8319
I20260806 22:27:20.469393 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:20.469945 1057 external_mini_cluster.cc:1428] Setting key c845ea7bf72c647e2811c1258a64d60f
Aug 06 22:27:20 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055240, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:20.490933 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.470445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:37271 (local address 127.1.8.126:45551)
0806 22:27:20.471041 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.471049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.474389 (+ 3340us) server_negotiation.cc:407] Connection header received
0806 22:27:20.474458 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.474462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.474527 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.474617 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:20.476486 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.477379 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.479127 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.479579 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.479699 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.483467 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:20.483472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:20.483478 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:20.483521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:20.487688 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.488632 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.488636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.488638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.488774 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.489198 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.489201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.489203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.489503 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:20.489630 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.490419 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.490609 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":303,"threads_started":1}
I20260806 22:27:20.492193 8525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.471027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45551 (local address 127.1.8.67:37271)
0806 22:27:20.472189 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.472239 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.474076 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.474875 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.474883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.475569 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:20.476329 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.476349 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.477524 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.477528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.478965 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.478977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.479790 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.479794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.479974 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.480717 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:20.480740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:20.483193 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:20.487878 (+ 4685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:20.487886 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:20.487901 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:20.488507 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:20.488870 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:20.488874 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:20.488876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:20.489094 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:20.489626 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:20.489632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:20.490005 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.490244 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.490689 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":660,"threads_started":1}
I20260806 22:27:20.493973 8521 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45551
I20260806 22:27:20.494338 8521 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:20.495291 8521 heartbeater.cc:511] Master 127.1.8.126:45551 requested a full tablet report, sending...
I20260806 22:27:20.497637 7800 ts_manager.cc:194] Registered new tserver with Master: 81999fd97b2a4279b7e6b023629b8384 (127.1.8.67:43751)
I20260806 22:27:20.498907 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:27:20.499192 7800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:37271
I20260806 22:27:20.520906 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.506299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.1.8.126:45551)
0806 22:27:20.506517 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.506523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.507166 (+ 643us) server_negotiation.cc:407] Connection header received
0806 22:27:20.507336 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.507340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.507440 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.507551 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:20.509405 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.510263 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.511682 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.512198 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.512306 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.514987 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:20.514992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:20.514995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:20.515035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:20.518057 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.518769 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.518772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.518774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.518932 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.519424 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.519427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.519429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.519791 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:20.520028 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.520434 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.520642 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:27:20.529996 7800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:20.537810 7800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:20.565482 8541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.553097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:46097 (local address 127.0.0.1:48508)
0806 22:27:20.556165 (+ 3068us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.556182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.556328 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.557455 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.557466 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.557515 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:20.558081 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.558098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.560207 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.560214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.562595 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.562615 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.564472 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.564478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.564687 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.564740 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.564871 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.565057 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2896,"thread_start_us":2737,"threads_started":1}
I20260806 22:27:20.565526 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.553877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.1.8.66:46097)
0806 22:27:20.555667 (+ 1790us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.555678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.556479 (+ 801us) server_negotiation.cc:407] Connection header received
0806 22:27:20.556607 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.556617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.556877 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.557108 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:20.558291 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.560051 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.562789 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.564267 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.564394 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.564612 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.564948 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.565116 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"thread_start_us":1434,"threads_started":1}
I20260806 22:27:20.566362 8545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.556258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60264)
0806 22:27:20.557030 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.557052 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.557197 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.558910 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.558915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.558942 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:20.559563 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.559576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.561280 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.561286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.563537 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.563552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.565426 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.565433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.565638 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.565683 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.565755 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.565876 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":8,"spinlock_wait_cycles":3840,"thread_start_us":410,"threads_started":1}
I20260806 22:27:20.567077 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.555854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.1.8.65:37027)
0806 22:27:20.558001 (+ 2147us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.558010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.558038 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:27:20.558125 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.558131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.558388 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.559820 (+ 1432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:20.560140 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.561140 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.563711 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.565165 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.566142 (+ 977us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.566383 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.566454 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.566612 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1916,"thread_start_us":1692,"threads_started":1}
I20260806 22:27:20.567807 8540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.552789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.1.8.67:43751)
0806 22:27:20.558387 (+ 5598us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.558393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.558416 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:27:20.558489 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.558494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.558716 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.559855 (+ 1139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:20.560152 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.561823 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.564205 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.565677 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.565793 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.566015 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.567287 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.567458 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5393,"thread_start_us":5194,"threads_started":1}
I20260806 22:27:20.568140 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.552435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43751 (local address 127.0.0.1:49980)
0806 22:27:20.554719 (+ 2284us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.554770 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.554954 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.559003 (+ 4049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.559012 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.559053 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:20.559664 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.559681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.561976 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.561982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.564007 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.564018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.566168 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.566178 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.566367 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.566401 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.566636 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.566794 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2022,"mutex_wait_us":24,"spinlock_wait_cycles":7936,"thread_start_us":2226,"threads_started":1}
I20260806 22:27:20.570288 8208 tablet_service.cc:1467] Processing CreateTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc (DEFAULT_TABLE table=test-table [id=705b7ae53580434384c43cf5f9227a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:20.571768 8208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4743b5a6627c47da8bb7bda34d6ea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:20.571244 7985 tablet_service.cc:1467] Processing CreateTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc (DEFAULT_TABLE table=test-table [id=705b7ae53580434384c43cf5f9227a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:20.571848 8411 tablet_service.cc:1467] Processing CreateTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc (DEFAULT_TABLE table=test-table [id=705b7ae53580434384c43cf5f9227a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:20.573020 7985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4743b5a6627c47da8bb7bda34d6ea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:20.573216 8411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4743b5a6627c47da8bb7bda34d6ea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:20.584018 8546 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Bootstrap starting.
I20260806 22:27:20.588323 8546 tablet_bootstrap.cc:654] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:20.588315 8547 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Bootstrap starting.
I20260806 22:27:20.589767 8546 log.cc:824] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:20.590831 8550 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Bootstrap starting.
I20260806 22:27:20.592361 8547 tablet_bootstrap.cc:654] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:20.593700 8547 log.cc:824] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:20.594358 8550 tablet_bootstrap.cc:654] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:20.595520 8546 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: No bootstrap required, opened a new log
I20260806 22:27:20.595803 8550 log.cc:824] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:20.595901 8546 ts_tablet_manager.cc:1397] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260806 22:27:20.603850 8550 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: No bootstrap required, opened a new log
I20260806 22:27:20.603850 8547 tablet_bootstrap.cc:492] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: No bootstrap required, opened a new log
I20260806 22:27:20.604205 8547 ts_tablet_manager.cc:1397] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260806 22:27:20.604204 8550 ts_tablet_manager.cc:1397] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260806 22:27:20.607499 8546 raft_consensus.cc:348] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.607906 8546 raft_consensus.cc:374] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:20.608029 8546 raft_consensus.cc:729] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f4f6b65d4457bb4607ee52fe1ef45, State: Initialized, Role: FOLLOWER
I20260806 22:27:20.608690 8546 consensus_queue.cc:260] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.609928 8312 heartbeater.cc:503] Master 127.1.8.126:45551 was elected leader, sending a full tablet report...
I20260806 22:27:20.610672 8546 ts_tablet_manager.cc:1428] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260806 22:27:20.615521 8550 raft_consensus.cc:348] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.615521 8547 raft_consensus.cc:348] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.615989 8547 raft_consensus.cc:374] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:20.616019 8550 raft_consensus.cc:374] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:20.616127 8550 raft_consensus.cc:729] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81999fd97b2a4279b7e6b023629b8384, State: Initialized, Role: FOLLOWER
I20260806 22:27:20.616134 8547 raft_consensus.cc:729] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b23a8ff67be449eacbd751056ee9660, State: Initialized, Role: FOLLOWER
I20260806 22:27:20.616771 8550 consensus_queue.cc:260] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.616771 8547 consensus_queue.cc:260] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.618665 8550 ts_tablet_manager.cc:1428] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 22:27:20.618672 8547 ts_tablet_manager.cc:1428] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260806 22:27:20.619439 8521 heartbeater.cc:503] Master 127.1.8.126:45551 was elected leader, sending a full tablet report...
W20260806 22:27:20.719107 8522 tablet.cc:2369] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:27:20.766651 8095 tablet.cc:2369] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:20.774161 8558 raft_consensus.cc:482] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:20.774444 8558 raft_consensus.cc:504] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.776319 8558 leader_election.cc:288] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81999fd97b2a4279b7e6b023629b8384 (127.1.8.67:43751), 7b23a8ff67be449eacbd751056ee9660 (127.1.8.65:37027)
I20260806 22:27:20.783164 8540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.776810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:49763 (local address 127.1.8.67:43751)
0806 22:27:20.777065 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.777071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.777387 (+ 316us) server_negotiation.cc:407] Connection header received
0806 22:27:20.777763 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.777769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.777850 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.777954 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:20.778569 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.779347 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.781381 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.782189 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.782344 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.782469 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.782786 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.782909 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:27:20.783354 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.776562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43751 (local address 127.1.8.66:49763)
0806 22:27:20.777272 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.777295 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.777511 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.777987 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.777991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.778026 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:20.778418 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.778428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.779487 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.779493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.781220 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.781233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.782471 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.782479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.782615 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.782639 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.782758 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.782891 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":47,"thread_start_us":341,"threads_started":1}
I20260806 22:27:20.783452 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.777520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:59851 (local address 127.1.8.65:37027)
0806 22:27:20.777765 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.777771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.778270 (+ 499us) server_negotiation.cc:407] Connection header received
0806 22:27:20.778483 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.778487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.778542 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.778642 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:20.779199 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.779912 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.781777 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.782505 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.782660 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.782768 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.783080 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.783204 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 22:27:20.783679 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.777283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.1.8.66:59851)
0806 22:27:20.778152 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:20.778167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:20.778336 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:20.778657 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:20.778661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:20.778683 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:20.779052 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.779061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.780058 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.780075 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.781626 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:20.781637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.782737 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.782743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:20.782905 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.782927 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:27:20.783046 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:27:20.783167 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":7,"thread_start_us":583,"threads_started":1}
I20260806 22:27:20.784628 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4743b5a6627c47da8bb7bda34d6ea5cc" candidate_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81999fd97b2a4279b7e6b023629b8384" is_pre_election: true
I20260806 22:27:20.784842 8025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4743b5a6627c47da8bb7bda34d6ea5cc" candidate_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b23a8ff67be449eacbd751056ee9660" is_pre_election: true
I20260806 22:27:20.785249 8449 raft_consensus.cc:2454] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90f4f6b65d4457bb4607ee52fe1ef45 in term 0.
I20260806 22:27:20.785413 8025 raft_consensus.cc:2454] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90f4f6b65d4457bb4607ee52fe1ef45 in term 0.
I20260806 22:27:20.786008 8125 leader_election.cc:302] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81999fd97b2a4279b7e6b023629b8384, e90f4f6b65d4457bb4607ee52fe1ef45; no voters:
I20260806 22:27:20.786394 8558 raft_consensus.cc:2781] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:20.786552 8558 raft_consensus.cc:482] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:20.786687 8558 raft_consensus.cc:3037] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:20.789000 8558 raft_consensus.cc:504] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.789637 8558 leader_election.cc:288] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [CANDIDATE]: Term 1 election: Requested vote from peers 81999fd97b2a4279b7e6b023629b8384 (127.1.8.67:43751), 7b23a8ff67be449eacbd751056ee9660 (127.1.8.65:37027)
I20260806 22:27:20.790055 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4743b5a6627c47da8bb7bda34d6ea5cc" candidate_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81999fd97b2a4279b7e6b023629b8384"
I20260806 22:27:20.790086 8025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4743b5a6627c47da8bb7bda34d6ea5cc" candidate_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b23a8ff67be449eacbd751056ee9660"
I20260806 22:27:20.790288 8449 raft_consensus.cc:3037] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:20.790302 8025 raft_consensus.cc:3037] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:20.792960 8449 raft_consensus.cc:2454] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90f4f6b65d4457bb4607ee52fe1ef45 in term 1.
I20260806 22:27:20.793032 8025 raft_consensus.cc:2454] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90f4f6b65d4457bb4607ee52fe1ef45 in term 1.
I20260806 22:27:20.793365 8125 leader_election.cc:302] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81999fd97b2a4279b7e6b023629b8384, e90f4f6b65d4457bb4607ee52fe1ef45; no voters:
I20260806 22:27:20.793715 8558 raft_consensus.cc:2781] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:20.794199 8558 raft_consensus.cc:686] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 1 LEADER]: Becoming Leader. State: Replica: e90f4f6b65d4457bb4607ee52fe1ef45, State: Running, Role: LEADER
I20260806 22:27:20.794705 8558 consensus_queue.cc:237] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } }
I20260806 22:27:20.799440 7802 catalog_manager.cc:5495] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90f4f6b65d4457bb4607ee52fe1ef45 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90f4f6b65d4457bb4607ee52fe1ef45" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46097 } health_report { overall_health: HEALTHY } } }
W20260806 22:27:20.848189 8313 tablet.cc:2369] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:20.863372 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.857853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.1.8.66:46097)
0806 22:27:20.858073 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.858101 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.858274 (+ 173us) server_negotiation.cc:407] Connection header received
0806 22:27:20.858525 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.858529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.858612 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.858800 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:20.859188 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.859949 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.861288 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.861690 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.861827 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.862292 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:20.862772 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:20.862929 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.863017 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.863110 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:27:20.871620 8025 raft_consensus.cc:1261] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 1 FOLLOWER]: Refusing update from remote peer e90f4f6b65d4457bb4607ee52fe1ef45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:20.871683 8449 raft_consensus.cc:1261] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 1 FOLLOWER]: Refusing update from remote peer e90f4f6b65d4457bb4607ee52fe1ef45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:20.872699 8558 consensus_queue.cc:1035] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81999fd97b2a4279b7e6b023629b8384" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:20.873030 8564 consensus_queue.cc:1035] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b23a8ff67be449eacbd751056ee9660" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:20.919703 7802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:20.920132 7802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:20.928314 7802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fda50f435d7a4b6f9b459a6442a0475a: Sending DeleteTablet for 3 replicas of tablet 4743b5a6627c47da8bb7bda34d6ea5cc
I20260806 22:27:20.929694 8208 tablet_service.cc:1514] Processing DeleteTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:20 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:48508
I20260806 22:27:20.930089 7985 tablet_service.cc:1514] Processing DeleteTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:20 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:60264
I20260806 22:27:20.930162 8411 tablet_service.cc:1514] Processing DeleteTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:20 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49980
I20260806 22:27:20.930975 7984 tablet_service.cc:1514] Processing DeleteTablet for tablet 4743b5a6627c47da8bb7bda34d6ea5cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:20 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:60264
I20260806 22:27:20.931555 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 7894
I20260806 22:27:20.956815 8580 tablet_replica.cc:330] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: stopping tablet replica
I20260806 22:27:20.958171 8579 tablet_replica.cc:330] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: stopping tablet replica
I20260806 22:27:20.958549 8579 raft_consensus.cc:2229] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:20.957566 8580 raft_consensus.cc:2229] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:20.959278 8580 raft_consensus.cc:2258] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:20.960184 8579 raft_consensus.cc:2258] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:20.962616 8580 ts_tablet_manager.cc:1905] T 4743b5a6627c47da8bb7bda34d6ea5cc P 7b23a8ff67be449eacbd751056ee9660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:20.966755 8579 ts_tablet_manager.cc:1905] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:27:20 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:20.972890 8581 tablet_replica.cc:330] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: stopping tablet replica
I20260806 22:27:20.973333 8581 raft_consensus.cc:2229] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:20.973958 8581 raft_consensus.cc:2258] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:20.975858 8581 ts_tablet_manager.cc:1905] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:20.978253 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:20.933776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.1.8.65:37027)
0806 22:27:20.934001 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:20.934010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:20.946949 (+ 12939us) server_negotiation.cc:407] Connection header received
0806 22:27:20.947504 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:20.947513 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:20.947652 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:20.947747 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:20.959517 (+ 11770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.960769 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.962691 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:20.963388 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:20.963490 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:20.969336 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:20.969351 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:20.969367 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:20.969416 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:20.974324 (+ 4908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.975765 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.975775 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.975779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.975964 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:20.976437 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:20.976442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:20.976446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:20.976908 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:20.977258 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:20.977565 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 22:27:20.977799 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:27:20.979080 8086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:20.983232 8581 ts_tablet_manager.cc:1918] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:20.983402 8581 log.cc:1196] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/4743b5a6627c47da8bb7bda34d6ea5cc
I20260806 22:27:20.983963 8581 ts_tablet_manager.cc:1939] T 4743b5a6627c47da8bb7bda34d6ea5cc P 81999fd97b2a4279b7e6b023629b8384: Deleting consensus metadata
I20260806 22:27:20.985746 7779 catalog_manager.cc:4841] TS 81999fd97b2a4279b7e6b023629b8384 (127.1.8.67:43751): tablet 4743b5a6627c47da8bb7bda34d6ea5cc (table test-table [id=705b7ae53580434384c43cf5f9227a43]) successfully deleted
I20260806 22:27:20.995810 8579 ts_tablet_manager.cc:1918] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:20.996031 8579 log.cc:1196] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/4743b5a6627c47da8bb7bda34d6ea5cc
I20260806 22:27:20.996865 8579 ts_tablet_manager.cc:1939] T 4743b5a6627c47da8bb7bda34d6ea5cc P e90f4f6b65d4457bb4607ee52fe1ef45: Deleting consensus metadata
I20260806 22:27:20.998855 7777 catalog_manager.cc:4841] TS e90f4f6b65d4457bb4607ee52fe1ef45 (127.1.8.66:46097): tablet 4743b5a6627c47da8bb7bda34d6ea5cc (table test-table [id=705b7ae53580434384c43cf5f9227a43]) successfully deleted
I20260806 22:27:21.080650 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 7894
I20260806 22:27:21.107051 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 8105
W20260806 22:27:21.107051 7779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:37027 (error 108)
I20260806 22:27:21.109354 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.108479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60280)
0806 22:27:21.108956 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.109113 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":281,"threads_started":1}
W20260806 22:27:21.109694 7779 catalog_manager.cc:4584] TS 7b23a8ff67be449eacbd751056ee9660 (127.1.8.65:37027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4743b5a6627c47da8bb7bda34d6ea5cc: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Aug 06 22:27:21 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:27:21.126543 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.108291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.1.8.66:46097)
0806 22:27:21.108482 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:21.108489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:21.109110 (+ 621us) server_negotiation.cc:407] Connection header received
0806 22:27:21.109317 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:21.109324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:21.109426 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:21.109532 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:21.111370 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:21.112262 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:21.114097 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:21.114538 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:21.114642 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:21.120317 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:21.120334 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:21.120351 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:21.120402 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:21.123403 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:21.124190 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:21.124198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:21.124200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:21.124357 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:21.124795 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:21.124799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:21.124801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:21.125207 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:21.125576 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:21.125869 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:27:21.126173 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:27:21.127334 8305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:21.154356 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.153814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60292)
0806 22:27:21.154044 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.154184 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 22:27:21.155475 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.155051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60296)
0806 22:27:21.155227 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.155310 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:27:21.163735 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.163327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60298)
0806 22:27:21.163501 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.163609 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:27:21.164759 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.164362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60310)
0806 22:27:21.164505 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.164603 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:27:21.204239 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.203641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60312)
0806 22:27:21.203884 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.203992 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:27:21.205384 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.204919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60326)
0806 22:27:21.205133 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.205224 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 22:27:21.241885 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.241306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60334)
0806 22:27:21.241593 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.241711 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 22:27:21.243209 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.242567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60340)
0806 22:27:21.242724 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.242833 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 22:27:21.244988 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 8105
I20260806 22:27:21.272817 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 8319
Aug 06 22:27:21 dist-test-slave-7vjv krb5kdc[7731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:27:21.287282 8540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.273705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49982 (local address 127.1.8.67:43751)
0806 22:27:21.273911 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:21.273917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:21.273940 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:27:21.274074 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:21.274078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:21.274150 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:21.274252 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:21.275520 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:21.276330 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:21.277502 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:21.277895 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:21.277988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:21.282358 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:21.282371 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:21.282387 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:21.282425 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:21.285031 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:21.285627 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:21.285634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:21.285637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:21.285783 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:21.286106 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:21.286109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:21.286110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:21.286415 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:21.286647 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:21.286860 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:27:21.287015 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:27:21.287950 8506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:21.306555 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.306120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60342)
0806 22:27:21.306322 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.306407 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:27:21.307626 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.307272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60348)
0806 22:27:21.307431 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.307498 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 22:27:21.345042 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.344555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60350)
0806 22:27:21.344799 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.344893 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 22:27:21.346160 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.345757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60356)
0806 22:27:21.345926 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.345993 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:27:21.378832 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 8319
I20260806 22:27:21.402927 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 7751
I20260806 22:27:21.404217 7848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:21.535881 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:21.535434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37027 (local address 127.0.0.1:60360)
0806 22:27:21.535658 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:21.535742 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:27:21.535974 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 7751
2026-08-06T22:27:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5655 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:21Z Disabled control of system clock
I20260806 22:27:21.624368 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33045
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:27:21.965628 8598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:21.965924 8598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:21.966009 8598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:21.973392 8598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:21.973539 8598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:21.973580 8598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:21.973690 8598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:21.973757 8598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:21.983660 8598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:21.985131 8598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:21.986996 8598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:21.999280 8607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:21.999320 8609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:21.999502 8611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:22.000239 8598 server_base.cc:1034] running on GCE node
I20260806 22:27:22.001037 8598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:22.002431 8598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:22.003675 8598 hybrid_clock.cc:648] HybridClock initialized: now 1786055242003637 us; error 52 us; skew 500 ppm
I20260806 22:27:22.004058 8598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:22.006660 8598 webserver.cc:460] Webserver started at http://127.1.8.126:42037/ using document root <none> and password file <none>
I20260806 22:27:22.007391 8598 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:22.007534 8598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:22.007871 8598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:22.010893 8598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "c6bbfe4afb154b7e9c830b2e78461fb6"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "72e5fcd1938ac9bafb0c17b7eed74507"
server_key_iv: "f76a00c0500a780f87d417b631d1fb30"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.011723 8598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "c6bbfe4afb154b7e9c830b2e78461fb6"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "72e5fcd1938ac9bafb0c17b7eed74507"
server_key_iv: "f76a00c0500a780f87d417b631d1fb30"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.016970 8598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:22.020533 8622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:22.021850 8598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:27:22.022042 8598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "c6bbfe4afb154b7e9c830b2e78461fb6"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "72e5fcd1938ac9bafb0c17b7eed74507"
server_key_iv: "f76a00c0500a780f87d417b631d1fb30"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.022310 8598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:22.028640 8598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:22.029585 8598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:22.029903 8598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:22.049947 8598 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:33045
I20260806 22:27:22.049976 8727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:33045 every 8 connection(s)
I20260806 22:27:22.051409 8598 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:22.055200 8728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:22.056559 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 8598
I20260806 22:27:22.056821 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:22.057508 1057 external_mini_cluster.cc:1428] Setting key 58cfd6fbb9a0e390d1263d9dc4fd6f2d
I20260806 22:27:22.066040 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:22.059639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.1.8.126:33045)
0806 22:27:22.060816 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:22.060829 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:22.060876 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:27:22.062552 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:22.062574 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:22.062960 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:22.063275 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:22.064459 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:22.064465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:22.064477 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:22.064492 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:22.064557 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:22.064576 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:22.064770 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:22.064872 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:27:22.065158 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":517,"threads_started":1}
I20260806 22:27:22.067227 8728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: Bootstrap starting.
I20260806 22:27:22.070307 8728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:22.071287 8728 log.cc:824] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:22.074196 8728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: No bootstrap required, opened a new log
I20260806 22:27:22.081297 8728 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } }
I20260806 22:27:22.081652 8728 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:22.081791 8728 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6bbfe4afb154b7e9c830b2e78461fb6, State: Initialized, Role: FOLLOWER
I20260806 22:27:22.082381 8728 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } }
I20260806 22:27:22.082636 8728 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:22.082780 8728 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:22.082952 8728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:22.085151 8728 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } }
I20260806 22:27:22.085638 8728 leader_election.cc:302] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6bbfe4afb154b7e9c830b2e78461fb6; no voters:
I20260806 22:27:22.086341 8728 leader_election.cc:288] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:22.086454 8734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:22.087216 8734 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [term 1 LEADER]: Becoming Leader. State: Replica: c6bbfe4afb154b7e9c830b2e78461fb6, State: Running, Role: LEADER
I20260806 22:27:22.087817 8734 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } }
I20260806 22:27:22.088693 8728 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:22.090672 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } } }
I20260806 22:27:22.090764 8736 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6bbfe4afb154b7e9c830b2e78461fb6. Latest consensus state: current_term: 1 leader_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bbfe4afb154b7e9c830b2e78461fb6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33045 } } }
I20260806 22:27:22.091148 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:22.091157 8736 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:22.094731 8741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:22.099220 8741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:22.108163 8741 catalog_manager.cc:1347] Generated new cluster ID: aaff029d8a1d44b99dd8bc3d1eabb4c1
I20260806 22:27:22.108285 8741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:22.143361 8741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:22.143887 8741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:22.151445 8741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: Generated new TSK 0
I20260806 22:27:22.152510 8741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:27:22.168610 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--builtin_ntp_servers=127.1.8.84:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:27:22.529337 8762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:22.529615 8762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:22.529717 8762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:22.537041 8762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:22.537140 8762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:22.537245 8762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:22.546126 8762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:22.547427 8762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:22.549243 8762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:22.562430 8773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:22.562404 8774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:22.562420 8777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:22.563423 8762 server_base.cc:1034] running on GCE node
I20260806 22:27:22.564056 8762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:22.565088 8762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:22.566423 8762 hybrid_clock.cc:648] HybridClock initialized: now 1786055242566359 us; error 51 us; skew 500 ppm
I20260806 22:27:22.566809 8762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:22.569339 8762 webserver.cc:460] Webserver started at http://127.1.8.65:36555/ using document root <none> and password file <none>
I20260806 22:27:22.570056 8762 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:22.570186 8762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:22.570533 8762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:22.573694 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "a567a5daa5704fb9aa35e4ff0f692e8c"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "feb035388fc6e49a2980458f698037c6"
server_key_iv: "04f9d87ade0a458977b48bc8fee4264e"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.574492 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "a567a5daa5704fb9aa35e4ff0f692e8c"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "feb035388fc6e49a2980458f698037c6"
server_key_iv: "04f9d87ade0a458977b48bc8fee4264e"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.579864 8762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:27:22.583277 8786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:22.584671 8762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:22.584846 8762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "a567a5daa5704fb9aa35e4ff0f692e8c"
format_stamp: "Formatted at 2026-08-06 22:27:22 on dist-test-slave-7vjv"
server_key: "feb035388fc6e49a2980458f698037c6"
server_key_iv: "04f9d87ade0a458977b48bc8fee4264e"
server_key_version: "encryptionkey@0"
I20260806 22:27:22.585088 8762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:22.591185 8762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:22.592033 8762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:22.592371 8762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:22.593297 8762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:22.594871 8762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:22.594987 8762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:22.595125 8762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:22.595227 8762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:22.629385 8762 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:36481
I20260806 22:27:22.629441 8985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:36481 every 8 connection(s)
I20260806 22:27:22.630769 8762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:22.638782 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:22.633413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60173 (local address 127.1.8.126:33045)
0806 22:27:22.633907 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:22.633914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:22.635071 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 22:27:22.636774 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:22.636779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:22.636836 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:22.636948 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:22.638152 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:22.638156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:22.638159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:22.638176 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:22.638220 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:22.638232 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:22.638333 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:22.638500 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:27:22.638553 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260806 22:27:22.639644 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:22.633998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33045 (local address 127.1.8.65:60173)
0806 22:27:22.634908 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:22.634951 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:22.636533 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:22.637145 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:22.637153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:22.637238 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:22.637866 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:22.637869 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:22.637905 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:22.637988 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:22.638317 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:22.638324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:22.638330 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:27:22.638535 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 22:27:22.638721 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":295,"threads_started":1}
I20260806 22:27:22.640339 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 8762
I20260806 22:27:22.640607 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:22.641274 8986 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33045
I20260806 22:27:22.641384 1057 external_mini_cluster.cc:1428] Setting key d49a1f12a5ecceb003aa6fa543aa1dec
I20260806 22:27:22.641582 8986 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:22.642212 8986 heartbeater.cc:511] Master 127.1.8.126:33045 requested a full tablet report, sending...
I20260806 22:27:22.644531 8670 ts_manager.cc:194] Registered new tserver with Master: a567a5daa5704fb9aa35e4ff0f692e8c (127.1.8.65:36481)
I20260806 22:27:22.644575 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--builtin_ntp_servers=127.1.8.84:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:27:22.982818 8995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:22.983160 8995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:22.983311 8995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:22.991111 8995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:22.991246 8995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:22.991422 8995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:23.001003 8995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:23.002432 8995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:23.004383 8995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:23.017993 9004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:23.018126 9003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:23.018129 9006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:23.018982 8995 server_base.cc:1034] running on GCE node
I20260806 22:27:23.019666 8995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:23.020711 8995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:23.021945 8995 hybrid_clock.cc:648] HybridClock initialized: now 1786055243021900 us; error 51 us; skew 500 ppm
I20260806 22:27:23.022281 8995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:23.024665 8995 webserver.cc:460] Webserver started at http://127.1.8.66:38639/ using document root <none> and password file <none>
I20260806 22:27:23.025413 8995 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:23.025548 8995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:23.025889 8995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:23.029032 8995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "24e2848b834b44ab96252f3ab71ba711"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "c57b807910f95edde8f4a8cf28d597a6"
server_key_iv: "cabb160222da22cf4d8be11d36c416b2"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.029942 8995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "24e2848b834b44ab96252f3ab71ba711"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "c57b807910f95edde8f4a8cf28d597a6"
server_key_iv: "cabb160222da22cf4d8be11d36c416b2"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.035507 8995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:27:23.039181 9016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.040573 8995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:23.040764 8995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "24e2848b834b44ab96252f3ab71ba711"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "c57b807910f95edde8f4a8cf28d597a6"
server_key_iv: "cabb160222da22cf4d8be11d36c416b2"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.041014 8995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:23.046895 8995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:23.047732 8995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:23.048019 8995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:23.049019 8995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:23.050683 8995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:23.050802 8995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.050937 8995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:23.051024 8995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.086942 8995 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:33535
I20260806 22:27:23.086988 9200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:33535 every 8 connection(s)
I20260806 22:27:23.088505 8995 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:23.090742 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 8995
I20260806 22:27:23.090950 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:23.091516 1057 external_mini_cluster.cc:1428] Setting key ef51aa533ad374f7c2de82e502ffbd8c
I20260806 22:27:23.094427 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--builtin_ntp_servers=127.1.8.84:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:27:23.097512 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.091412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:40197 (local address 127.1.8.126:33045)
0806 22:27:23.091626 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.091633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.093260 (+ 1627us) server_negotiation.cc:407] Connection header received
0806 22:27:23.095029 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.095035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.095125 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.095241 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.096695 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.096701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.096706 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.096734 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.096769 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.096781 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.096927 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.097145 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.097222 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:27:23.098502 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.091919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33045 (local address 127.1.8.66:40197)
0806 22:27:23.093076 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.093131 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.094763 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.095477 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.095485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.095581 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.096377 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.096380 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.096421 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.096510 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.096897 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.096908 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.096916 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.097221 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.097495 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":434,"threads_started":1}
I20260806 22:27:23.100057 9201 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33045
I20260806 22:27:23.100414 9201 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:23.101038 9201 heartbeater.cc:511] Master 127.1.8.126:33045 requested a full tablet report, sending...
I20260806 22:27:23.102619 8670 ts_manager.cc:194] Registered new tserver with Master: 24e2848b834b44ab96252f3ab71ba711 (127.1.8.66:33535)
W20260806 22:27:23.437808 9205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:23.438134 9205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:23.438297 9205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:23.445755 9205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:23.445895 9205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:23.446081 9205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:23.455077 9205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:23.456491 9205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:23.458282 9205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:23.471860 9215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:23.471841 9217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:23.471922 9222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:23.472922 9205 server_base.cc:1034] running on GCE node
I20260806 22:27:23.473717 9205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:23.474721 9205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:23.475968 9205 hybrid_clock.cc:648] HybridClock initialized: now 1786055243475929 us; error 49 us; skew 500 ppm
I20260806 22:27:23.476353 9205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:23.478780 9205 webserver.cc:460] Webserver started at http://127.1.8.67:43673/ using document root <none> and password file <none>
I20260806 22:27:23.479489 9205 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:23.479637 9205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:23.480062 9205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:23.483091 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "bb70ca022e97bcc1b7db43fed35135c2"
server_key_iv: "bd335098725e191d8e6b724ca72abfa5"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.483925 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "bb70ca022e97bcc1b7db43fed35135c2"
server_key_iv: "bd335098725e191d8e6b724ca72abfa5"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.489099 9205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:27:23.492940 9231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.494247 9205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:23.494428 9205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5"
format_stamp: "Formatted at 2026-08-06 22:27:23 on dist-test-slave-7vjv"
server_key: "bb70ca022e97bcc1b7db43fed35135c2"
server_key_iv: "bd335098725e191d8e6b724ca72abfa5"
server_key_version: "encryptionkey@0"
I20260806 22:27:23.494676 9205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:23.500483 9205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:23.501235 9205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:23.501505 9205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:23.502386 9205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:23.503742 9205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:23.503872 9205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.503998 9205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:23.504089 9205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:23.539392 9205 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:38529
I20260806 22:27:23.539439 9404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:38529 every 8 connection(s)
I20260806 22:27:23.540874 9205 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:23.541271 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 9205
I20260806 22:27:23.541467 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:23.542126 1057 external_mini_cluster.cc:1428] Setting key 915ae02804bd96eb9df169d4f97b1fe8
I20260806 22:27:23.549057 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.543544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:42729 (local address 127.1.8.126:33045)
0806 22:27:23.543742 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.543749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.545485 (+ 1736us) server_negotiation.cc:407] Connection header received
0806 22:27:23.547076 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.547081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.547143 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.547227 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.548463 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.548467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.548470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.548486 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.548517 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.548529 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.548624 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.548793 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.548843 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:27:23.549919 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.544169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33045 (local address 127.1.8.67:42729)
0806 22:27:23.545311 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.545365 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.546850 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.547418 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.547425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.547509 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.548196 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.548198 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.548234 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.548316 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.548624 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.548631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.548637 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.548839 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.549021 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":398,"threads_started":1}
I20260806 22:27:23.551182 9405 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33045
I20260806 22:27:23.551460 9405 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:23.552047 9405 heartbeater.cc:511] Master 127.1.8.126:33045 requested a full tablet report, sending...
I20260806 22:27:23.553531 8670 ts_manager.cc:194] Registered new tserver with Master: 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 (127.1.8.67:38529)
I20260806 22:27:23.558094 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:27:23.565454 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.562739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.1.8.126:33045)
0806 22:27:23.562945 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.562954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.563360 (+ 406us) server_negotiation.cc:407] Connection header received
0806 22:27:23.563573 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.563581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.563642 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.563750 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.564807 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.564812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.564817 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.564836 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.564879 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.564898 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.565030 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.565120 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.565187 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:27:23.569542 8670 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:23.576468 8670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:23.595044 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.588597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:33535 (local address 127.0.0.1:49586)
0806 22:27:23.589728 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.589777 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.589947 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.592237 (+ 2290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.592244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.592326 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.593615 (+ 1289us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.593619 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.593649 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.593731 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.594275 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.594285 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.594293 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.594472 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.594565 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":579,"threads_started":1}
I20260806 22:27:23.595423 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.588882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.1.8.66:33535)
0806 22:27:23.590336 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.590345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.590377 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:27:23.590452 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.590462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.590719 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.592213 (+ 1494us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.593913 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.593922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.593945 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.593990 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.594085 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.594116 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.594438 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.594592 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.594779 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1281,"thread_start_us":1044,"threads_started":1}
I20260806 22:27:23.595872 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.588755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33600)
0806 22:27:23.591270 (+ 2515us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.591310 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.591423 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.592504 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.592511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.592536 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.593901 (+ 1365us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.593904 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.593919 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.593957 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.595286 (+ 1329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.595293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.595301 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.595429 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.595480 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2381,"thread_start_us":2232,"threads_started":1}
I20260806 22:27:23.596908 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.589340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.1.8.65:36481)
0806 22:27:23.591964 (+ 2624us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.591970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.591993 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:27:23.592045 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.592051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.592302 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.594117 (+ 1815us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.595029 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.595038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.595051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.595078 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.595132 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.595152 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.596368 (+ 1216us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.596440 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.596532 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2466,"thread_start_us":2326,"threads_started":1}
I20260806 22:27:23.600431 9100 tablet_service.cc:1467] Processing CreateTablet for tablet e65e8a0a685243459e4fb0b872d856af (DEFAULT_TABLE table=test-table [id=a704c52da841414cb2423c937e64c0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:23.602409 9100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65e8a0a685243459e4fb0b872d856af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:23.602546 8865 tablet_service.cc:1467] Processing CreateTablet for tablet e65e8a0a685243459e4fb0b872d856af (DEFAULT_TABLE table=test-table [id=a704c52da841414cb2423c937e64c0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:23.603945 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.591068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:38529 (local address 127.0.0.1:59132)
0806 22:27:23.591670 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.591690 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.591781 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.600863 (+ 9082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.600868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.600891 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.601628 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.601631 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.601648 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.601697 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.603296 (+ 1599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.603301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.603305 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.603436 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.603493 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":761,"threads_started":1}
I20260806 22:27:23.603978 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65e8a0a685243459e4fb0b872d856af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:23.604681 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.591799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.1.8.67:38529)
0806 22:27:23.599284 (+ 7485us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.599291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.599327 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:27:23.599403 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.599410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.599631 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.599791 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.602941 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.602950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.602964 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.602984 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.603040 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.603062 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.603237 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.604198 (+ 961us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.604298 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7243,"thread_start_us":7044,"threads_started":1}
I20260806 22:27:23.608336 9302 tablet_service.cc:1467] Processing CreateTablet for tablet e65e8a0a685243459e4fb0b872d856af (DEFAULT_TABLE table=test-table [id=a704c52da841414cb2423c937e64c0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:23.609881 9302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65e8a0a685243459e4fb0b872d856af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:23.615159 9441 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Bootstrap starting.
I20260806 22:27:23.618288 9441 tablet_bootstrap.cc:654] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:23.619313 9441 log.cc:824] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:23.620237 9442 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Bootstrap starting.
I20260806 22:27:23.623833 9442 tablet_bootstrap.cc:654] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:23.624001 9441 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: No bootstrap required, opened a new log
I20260806 22:27:23.624348 9441 ts_tablet_manager.cc:1397] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:27:23.624346 9444 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Bootstrap starting.
I20260806 22:27:23.625370 9442 log.cc:824] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:23.627926 9444 tablet_bootstrap.cc:654] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:23.629233 9444 log.cc:824] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:23.632333 9442 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: No bootstrap required, opened a new log
I20260806 22:27:23.632776 9442 ts_tablet_manager.cc:1397] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.002s
I20260806 22:27:23.634971 9441 raft_consensus.cc:348] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.635515 9441 raft_consensus.cc:374] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:23.635659 9441 raft_consensus.cc:729] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e2848b834b44ab96252f3ab71ba711, State: Initialized, Role: FOLLOWER
I20260806 22:27:23.636426 9441 consensus_queue.cc:260] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.647567 8986 heartbeater.cc:503] Master 127.1.8.126:33045 was elected leader, sending a full tablet report...
I20260806 22:27:23.647819 9444 tablet_bootstrap.cc:492] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: No bootstrap required, opened a new log
I20260806 22:27:23.648234 9444 ts_tablet_manager.cc:1397] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.007s
I20260806 22:27:23.650384 9442 raft_consensus.cc:348] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.650897 9442 raft_consensus.cc:374] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:23.651037 9442 raft_consensus.cc:729] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a567a5daa5704fb9aa35e4ff0f692e8c, State: Initialized, Role: FOLLOWER
I20260806 22:27:23.651706 9442 consensus_queue.cc:260] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.655117 9201 heartbeater.cc:503] Master 127.1.8.126:33045 was elected leader, sending a full tablet report...
I20260806 22:27:23.655869 9441 ts_tablet_manager.cc:1428] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Time spent starting tablet: real 0.031s user 0.017s sys 0.012s
I20260806 22:27:23.658419 9442 ts_tablet_manager.cc:1428] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Time spent starting tablet: real 0.026s user 0.008s sys 0.012s
I20260806 22:27:23.660691 9444 raft_consensus.cc:348] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.661226 9444 raft_consensus.cc:374] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:23.661352 9444 raft_consensus.cc:729] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc4ef2d2e6d4f1a881c2d2f776d7ce5, State: Initialized, Role: FOLLOWER
I20260806 22:27:23.662016 9444 consensus_queue.cc:260] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.663074 9405 heartbeater.cc:503] Master 127.1.8.126:33045 was elected leader, sending a full tablet report...
I20260806 22:27:23.663618 9444 ts_tablet_manager.cc:1428] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 22:27:23.789656 9448 raft_consensus.cc:482] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:23.790014 9448 raft_consensus.cc:504] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.791921 9448 leader_election.cc:288] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24e2848b834b44ab96252f3ab71ba711 (127.1.8.66:33535), 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 (127.1.8.67:38529)
W20260806 22:27:23.792703 9406 tablet.cc:2369] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:23.795621 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.792298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:40133 (local address 127.1.8.66:33535)
0806 22:27:23.792540 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.792546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.793204 (+ 658us) server_negotiation.cc:407] Connection header received
0806 22:27:23.793499 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.793509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.793612 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.793701 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.794944 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.794950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.794955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.794973 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.795013 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.795024 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.795160 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.795273 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.795343 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:27:23.795843 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.792292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:33535 (local address 127.1.8.65:40133)
0806 22:27:23.793094 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.793116 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.793307 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.793933 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.793937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.793956 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.794727 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.794729 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.794744 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.794786 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.795123 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.795127 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.795132 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.795295 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.795346 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":442,"threads_started":1}
I20260806 22:27:23.796226 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.793520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60777 (local address 127.1.8.67:38529)
0806 22:27:23.793760 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.793767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.794144 (+ 377us) server_negotiation.cc:407] Connection header received
0806 22:27:23.794342 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.794346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.794394 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.794500 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.795460 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.795466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.795471 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.795493 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.795529 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.795550 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.795686 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.795865 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.795958 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:27:23.796975 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.793257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:38529 (local address 127.1.8.65:60777)
0806 22:27:23.794017 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:23.794032 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:23.794196 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:23.794523 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:23.794528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:23.794552 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:23.795252 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:27:23.795255 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:23.795267 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:27:23.795307 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:23.795653 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:23.795658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:23.795666 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:27:23.796491 (+ 825us) client_negotiation.cc:240] Negotiation successful
0806 22:27:23.796609 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":465,"threads_started":1}
I20260806 22:27:23.797057 9135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65e8a0a685243459e4fb0b872d856af" candidate_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24e2848b834b44ab96252f3ab71ba711" is_pre_election: true
I20260806 22:27:23.797676 9135 raft_consensus.cc:2454] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a567a5daa5704fb9aa35e4ff0f692e8c in term 0.
I20260806 22:27:23.798038 9335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65e8a0a685243459e4fb0b872d856af" candidate_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" is_pre_election: true
I20260806 22:27:23.798508 8792 leader_election.cc:302] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e2848b834b44ab96252f3ab71ba711, a567a5daa5704fb9aa35e4ff0f692e8c; no voters:
I20260806 22:27:23.798624 9335 raft_consensus.cc:2454] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a567a5daa5704fb9aa35e4ff0f692e8c in term 0.
I20260806 22:27:23.798928 9448 raft_consensus.cc:2781] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:23.799079 9448 raft_consensus.cc:482] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:23.799176 9448 raft_consensus.cc:3037] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:23.801472 9448 raft_consensus.cc:504] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.802006 9448 leader_election.cc:288] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [CANDIDATE]: Term 1 election: Requested vote from peers 24e2848b834b44ab96252f3ab71ba711 (127.1.8.66:33535), 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 (127.1.8.67:38529)
I20260806 22:27:23.802512 9135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65e8a0a685243459e4fb0b872d856af" candidate_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24e2848b834b44ab96252f3ab71ba711"
I20260806 22:27:23.802573 9335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65e8a0a685243459e4fb0b872d856af" candidate_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5"
I20260806 22:27:23.802719 9135 raft_consensus.cc:3037] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:23.802794 9335 raft_consensus.cc:3037] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:23.805105 9135 raft_consensus.cc:2454] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a567a5daa5704fb9aa35e4ff0f692e8c in term 1.
I20260806 22:27:23.805200 9335 raft_consensus.cc:2454] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a567a5daa5704fb9aa35e4ff0f692e8c in term 1.
I20260806 22:27:23.805514 8792 leader_election.cc:302] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e2848b834b44ab96252f3ab71ba711, a567a5daa5704fb9aa35e4ff0f692e8c; no voters:
I20260806 22:27:23.805789 9448 raft_consensus.cc:2781] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:23.806243 9448 raft_consensus.cc:686] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 1 LEADER]: Becoming Leader. State: Replica: a567a5daa5704fb9aa35e4ff0f692e8c, State: Running, Role: LEADER
I20260806 22:27:23.806721 9448 consensus_queue.cc:237] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } }
I20260806 22:27:23.811203 8670 catalog_manager.cc:5495] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to a567a5daa5704fb9aa35e4ff0f692e8c (127.1.8.65). New cstate: current_term: 1 leader_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a567a5daa5704fb9aa35e4ff0f692e8c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 } health_report { overall_health: UNKNOWN } } }
I20260806 22:27:23.836217 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.834148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.1.8.65:36481)
0806 22:27:23.834310 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.834315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.834440 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:27:23.834621 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.834625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.834706 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.834788 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.835643 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.835647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.835651 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.835666 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.835698 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.835709 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.835796 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.835884 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.835932 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 22:27:23.840932 9202 tablet.cc:2369] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:23.844902 9135 raft_consensus.cc:1261] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 1 FOLLOWER]: Refusing update from remote peer a567a5daa5704fb9aa35e4ff0f692e8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:23.844903 9335 raft_consensus.cc:1261] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 1 FOLLOWER]: Refusing update from remote peer a567a5daa5704fb9aa35e4ff0f692e8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:23.845880 9457 consensus_queue.cc:1035] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e2848b834b44ab96252f3ab71ba711" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:23.846277 9448 consensus_queue.cc:1035] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc4ef2d2e6d4f1a881c2d2f776d7ce5" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:23.873267 8668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:23.873553 8668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:23.877307 8668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6bbfe4afb154b7e9c830b2e78461fb6: Sending DeleteTablet for 3 replicas of tablet e65e8a0a685243459e4fb0b872d856af
I20260806 22:27:23.878568 9100 tablet_service.cc:1514] Processing DeleteTablet for tablet e65e8a0a685243459e4fb0b872d856af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:23 UTC) from {username='slave'} at 127.0.0.1:49586
I20260806 22:27:23.878834 8865 tablet_service.cc:1514] Processing DeleteTablet for tablet e65e8a0a685243459e4fb0b872d856af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:23 UTC) from {username='slave'} at 127.0.0.1:33600
I20260806 22:27:23.880002 9302 tablet_service.cc:1514] Processing DeleteTablet for tablet e65e8a0a685243459e4fb0b872d856af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:23 UTC) from {username='slave'} at 127.0.0.1:59132
I20260806 22:27:23.880350 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 8762
I20260806 22:27:23.888819 9479 tablet_replica.cc:330] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: stopping tablet replica
I20260806 22:27:23.889405 9479 raft_consensus.cc:2229] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:23.890216 9479 raft_consensus.cc:2258] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:23.891094 9477 tablet_replica.cc:330] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: stopping tablet replica
I20260806 22:27:23.891544 9477 raft_consensus.cc:2229] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:23.892302 9477 raft_consensus.cc:2258] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:23.893160 9479 ts_tablet_manager.cc:1905] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:23.894464 9477 ts_tablet_manager.cc:1905] T e65e8a0a685243459e4fb0b872d856af P a567a5daa5704fb9aa35e4ff0f692e8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:23.895027 9476 tablet_replica.cc:330] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: stopping tablet replica
I20260806 22:27:23.895466 9476 raft_consensus.cc:2229] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:23.895996 9476 raft_consensus.cc:2258] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:23.897521 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:23.881889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.1.8.65:36481)
0806 22:27:23.882093 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:23.882099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:23.893392 (+ 11293us) server_negotiation.cc:407] Connection header received
0806 22:27:23.893756 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:23.893763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:23.893832 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:23.893912 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:23.896717 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:23.896723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:23.896728 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:23.896750 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:23.896792 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:23.896808 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:23.896908 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:23.897153 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:27:23.897220 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:27:23.898722 8980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:23.900380 9476 ts_tablet_manager.cc:1905] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:23.902056 9479 ts_tablet_manager.cc:1918] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:23.902544 9479 log.cc:1196] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/e65e8a0a685243459e4fb0b872d856af
I20260806 22:27:23.903350 9479 ts_tablet_manager.cc:1939] T e65e8a0a685243459e4fb0b872d856af P 9fc4ef2d2e6d4f1a881c2d2f776d7ce5: Deleting consensus metadata
I20260806 22:27:23.905804 8642 catalog_manager.cc:4841] TS 9fc4ef2d2e6d4f1a881c2d2f776d7ce5 (127.1.8.67:38529): tablet e65e8a0a685243459e4fb0b872d856af (table test-table [id=a704c52da841414cb2423c937e64c0c8]) successfully deleted
I20260806 22:27:23.907598 9476 ts_tablet_manager.cc:1918] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:23.907757 9476 log.cc:1196] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/e65e8a0a685243459e4fb0b872d856af
I20260806 22:27:23.908303 9476 ts_tablet_manager.cc:1939] T e65e8a0a685243459e4fb0b872d856af P 24e2848b834b44ab96252f3ab71ba711: Deleting consensus metadata
I20260806 22:27:23.909781 8642 catalog_manager.cc:4841] TS 24e2848b834b44ab96252f3ab71ba711 (127.1.8.66:33535): tablet e65e8a0a685243459e4fb0b872d856af (table test-table [id=a704c52da841414cb2423c937e64c0c8]) successfully deleted
I20260806 22:27:24.001384 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 8762
W20260806 22:27:24.030654 8623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:36481 (error 108)
I20260806 22:27:24.030843 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 8995
I20260806 22:27:24.032761 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.032214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33622)
0806 22:27:24.032394 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.032584 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:27:24.033003 8623 catalog_manager.cc:4584] TS a567a5daa5704fb9aa35e4ff0f692e8c (127.1.8.65:36481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e65e8a0a685243459e4fb0b872d856af: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
I20260806 22:27:24.034089 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.031698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.1.8.66:33535)
0806 22:27:24.031872 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:24.031877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:24.031973 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:27:24.032207 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:24.032214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:24.032330 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:24.032467 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:24.033502 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:24.033506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:24.033509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:24.033524 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:24.033555 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:24.033567 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:24.033673 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:24.033737 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:27:24.033784 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:27:24.034703 9193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:24.086661 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.086119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33624)
0806 22:27:24.086351 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.086480 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:27:24.087740 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.087365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33626)
0806 22:27:24.087518 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.087601 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:27:24.137846 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 8995
I20260806 22:27:24.148408 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.147779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33636)
0806 22:27:24.148050 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.148207 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 22:27:24.149564 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.149124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33642)
0806 22:27:24.149302 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.149398 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:27:24.162243 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 9205
I20260806 22:27:24.165095 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.163047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59134 (local address 127.1.8.67:38529)
0806 22:27:24.163248 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:24.163252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:24.163272 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:27:24.163376 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:24.163380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:24.163460 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:24.163546 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:24.164439 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:24.164443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:24.164447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:24.164463 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:24.164496 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:24.164508 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:24.164621 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:24.164695 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:27:24.164744 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:27:24.165751 9399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:24.230484 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.229977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33648)
0806 22:27:24.230211 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.230309 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:27:24.231604 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.231222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:36481 (local address 127.0.0.1:33656)
0806 22:27:24.231411 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:24.231482 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 22:27:24.256665 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 9205
I20260806 22:27:24.285099 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 8598
I20260806 22:27:24.286242 8721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:24.398764 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 8598
2026-08-06T22:27:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2864 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:27:24.453274 1057 mini_oidc.cc:152] Starting JWKS server
I20260806 22:27:24.456717 1057 webserver.cc:460] Webserver started at https://127.0.0.1:42237/ using document root <none> and password file <none>
I20260806 22:27:24.457051 1057 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:27:24.457814 1057 webserver.cc:460] Webserver started at http://127.0.0.1:37307/ using document root <none> and password file <none>
2026-08-06T22:27:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:24Z Disabled control of system clock
I20260806 22:27:24.495599 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40495
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42237/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/testchainca.cert with env {}
W20260806 22:27:24.831223 9497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:24.831558 9497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:24.831712 9497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:24.838903 9497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:24.839051 9497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:24.839161 9497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:24.839273 9497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:24.839363 9497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:27:24.839457 9497 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:27:24.839537 9497 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42237/jwks.json
I20260806 22:27:24.849644 9497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42237/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:24.851018 9497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:24.852370 9497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:24.864455 9504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:24.864506 9503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:24.864537 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:24.865568 9497 server_base.cc:1034] running on GCE node
I20260806 22:27:24.866364 9497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:24.867702 9497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:24.868945 9497 hybrid_clock.cc:648] HybridClock initialized: now 1786055244868907 us; error 41 us; skew 500 ppm
I20260806 22:27:24.869323 9497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:24.871567 9497 webserver.cc:460] Webserver started at http://127.1.8.126:41469/ using document root <none> and password file <none>
I20260806 22:27:24.872361 9497 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:24.872511 9497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:24.872831 9497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:24.875761 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "43b7be01255344e2b4c7ce968eabbf51"
format_stamp: "Formatted at 2026-08-06 22:27:24 on dist-test-slave-7vjv"
server_key: "3c4c171b93fd7e57d8cd43ebaab66218"
server_key_iv: "98cf575a63c9bab6424c3102a87c87e9"
server_key_version: "encryptionkey@0"
I20260806 22:27:24.876644 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "43b7be01255344e2b4c7ce968eabbf51"
format_stamp: "Formatted at 2026-08-06 22:27:24 on dist-test-slave-7vjv"
server_key: "3c4c171b93fd7e57d8cd43ebaab66218"
server_key_iv: "98cf575a63c9bab6424c3102a87c87e9"
server_key_version: "encryptionkey@0"
I20260806 22:27:24.881850 9497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:24.885161 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:24.886483 9497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:27:24.886691 9497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "43b7be01255344e2b4c7ce968eabbf51"
format_stamp: "Formatted at 2026-08-06 22:27:24 on dist-test-slave-7vjv"
server_key: "3c4c171b93fd7e57d8cd43ebaab66218"
server_key_iv: "98cf575a63c9bab6424c3102a87c87e9"
server_key_version: "encryptionkey@0"
I20260806 22:27:24.886929 9497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:24.912604 9497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:24.913482 9497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:24.913810 9497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:24.932796 9497 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40495
I20260806 22:27:24.932814 9599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40495 every 8 connection(s)
I20260806 22:27:24.934396 9497 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:24.937058 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 9497
I20260806 22:27:24.937266 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:24.937950 1057 external_mini_cluster.cc:1428] Setting key 16663d31b9d7547df2e769c1809c4832
I20260806 22:27:24.938297 9601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:24.953378 9601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51: Bootstrap starting.
I20260806 22:27:24.953400 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:24.940029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.1.8.126:40495)
0806 22:27:24.941088 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:24.941102 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:24.941156 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:27:24.943241 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:24.943268 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:24.943840 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:24.944256 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:24.944789 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:24.946837 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:24.948315 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:24.949331 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:24.949507 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:24.950675 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:24.950685 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:24.950709 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:24.951416 (+ 707us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:24.951508 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:24.951532 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:24.951656 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:24.951831 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:27:24.952170 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":324,"threads_started":1}
I20260806 22:27:24.956681 9601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:24.957075 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:27:24.957696 9601 log.cc:824] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:24.960538 9601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51: No bootstrap required, opened a new log
I20260806 22:27:24.967511 9601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } }
I20260806 22:27:24.967873 9601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:24.967989 9601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b7be01255344e2b4c7ce968eabbf51, State: Initialized, Role: FOLLOWER
I20260806 22:27:24.968608 9601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } }
I20260806 22:27:24.968830 9601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:24.968959 9601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:24.969184 9601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:24.971215 9601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } }
I20260806 22:27:24.971717 9601 leader_election.cc:302] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b7be01255344e2b4c7ce968eabbf51; no voters:
I20260806 22:27:24.972373 9601 leader_election.cc:288] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:24.972487 9610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:24.975205 9610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [term 1 LEADER]: Becoming Leader. State: Replica: 43b7be01255344e2b4c7ce968eabbf51, State: Running, Role: LEADER
I20260806 22:27:24.975713 9610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } }
I20260806 22:27:24.976217 9601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:24.978701 9613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b7be01255344e2b4c7ce968eabbf51. Latest consensus state: current_term: 1 leader_uuid: "43b7be01255344e2b4c7ce968eabbf51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } } }
I20260806 22:27:24.978924 9612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b7be01255344e2b4c7ce968eabbf51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b7be01255344e2b4c7ce968eabbf51" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40495 } } }
I20260806 22:27:24.979243 9613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:24.979272 9612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:24.981223 9622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:24.985885 9622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:24.993947 9622 catalog_manager.cc:1347] Generated new cluster ID: 0c180b3001bb4d989d35f70cf15cacc7
I20260806 22:27:24.994060 9622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:25.020437 9622 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:25.021826 9622 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:25.027146 9622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b7be01255344e2b4c7ce968eabbf51: Generated new TSK 0
I20260806 22:27:25.028204 9622 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:27:25.089856 9643 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:27:25.090082 9643 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:27:25.106009 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:25.054502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.1.8.126:40495)
0806 22:27:25.054771 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:25.054781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:25.055216 (+ 435us) server_negotiation.cc:407] Connection header received
0806 22:27:25.055577 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:25.055582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:25.055650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:25.055775 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:27:25.056453 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.057136 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.059211 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.059664 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.059763 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:25.060414 (+ 651us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:27:25.105215 (+ 44801us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:27:25.105665 (+ 450us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 22:27:25.106172 9607 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:27:25.107740 9641 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:25.054405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40495 (local address 127.0.0.1:60294)
0806 22:27:25.055106 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:25.055126 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:25.055315 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:25.055825 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:25.055830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:25.055844 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:27:25.056287 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:25.056300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.057271 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.057276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:25.059058 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:25.059075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.059789 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.059854 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:25.060030 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:25.060130 (+ 100us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:27:25.105617 (+ 45487us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:27:25.105927 (+ 310us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.8.126:40495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":333,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.8.126:40495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:27:25.109866 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 9497
I20260806 22:27:25.111047 9594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:25.208621 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 9497
2026-08-06T22:27:25Z chronyd exiting
I20260806 22:27:25.270594 1057 test_util.cc:175] -----------------------------------------------
I20260806 22:27:25.270716 1057 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055191034056-1057-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (822 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:27:25.275830 1057 mini_oidc.cc:152] Starting JWKS server
I20260806 22:27:25.277906 1057 webserver.cc:460] Webserver started at https://127.0.0.1:37727/ using document root <none> and password file <none>
I20260806 22:27:25.278170 1057 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:27:25.278805 1057 webserver.cc:460] Webserver started at http://127.0.0.1:37483/ using document root <none> and password file <none>
2026-08-06T22:27:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:25Z Disabled control of system clock
I20260806 22:27:25.316865 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40055
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:36975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40055
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37727/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/testchainca.cert with env {}
W20260806 22:27:25.649320 9652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:25.649643 9652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:25.649864 9652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:25.656850 9652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:25.656985 9652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:25.657076 9652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:25.657192 9652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:25.657292 9652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:27:25.657383 9652 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:27:25.657454 9652 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37727/jwks.json
I20260806 22:27:25.667105 9652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37727/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:25.668432 9652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:25.669708 9652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:25.681990 9664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:25.682039 9662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:25.682113 9661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:25.683069 9652 server_base.cc:1034] running on GCE node
I20260806 22:27:25.683938 9652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:25.685320 9652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:25.686525 9652 hybrid_clock.cc:648] HybridClock initialized: now 1786055245686490 us; error 24 us; skew 500 ppm
I20260806 22:27:25.686854 9652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:25.692655 9652 webserver.cc:460] Webserver started at http://127.1.8.126:37433/ using document root <none> and password file <none>
I20260806 22:27:25.693363 9652 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:25.693491 9652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:25.693853 9652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:25.696797 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5"
format_stamp: "Formatted at 2026-08-06 22:27:25 on dist-test-slave-7vjv"
server_key: "028799e49198ebfb9c10e3f47f881e47"
server_key_iv: "4dc36a19aa336ca530368a4240ea8a39"
server_key_version: "encryptionkey@0"
I20260806 22:27:25.697575 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5"
format_stamp: "Formatted at 2026-08-06 22:27:25 on dist-test-slave-7vjv"
server_key: "028799e49198ebfb9c10e3f47f881e47"
server_key_iv: "4dc36a19aa336ca530368a4240ea8a39"
server_key_version: "encryptionkey@0"
I20260806 22:27:25.702656 9652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:25.705960 9676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:25.707252 9652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:25.707443 9652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5"
format_stamp: "Formatted at 2026-08-06 22:27:25 on dist-test-slave-7vjv"
server_key: "028799e49198ebfb9c10e3f47f881e47"
server_key_iv: "4dc36a19aa336ca530368a4240ea8a39"
server_key_version: "encryptionkey@0"
I20260806 22:27:25.707690 9652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:25.724388 9652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:25.725180 9652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:25.725466 9652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:25.743652 9652 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40055
I20260806 22:27:25.743686 9763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40055 every 8 connection(s)
I20260806 22:27:25.745051 9652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:25.748669 9764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:25.749188 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 9652
I20260806 22:27:25.749398 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:25.750036 1057 external_mini_cluster.cc:1428] Setting key 28adb3cebbb2c1d1b63ac9de55a2346d
I20260806 22:27:25.763312 9764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5: Bootstrap starting.
I20260806 22:27:25.765380 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:25.752033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.1.8.126:40055)
0806 22:27:25.753159 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:25.753173 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:25.753234 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:27:25.755399 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:25.755426 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:25.756017 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:25.756476 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:25.756996 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.758801 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.760083 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.761193 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.761371 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:25.762513 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:25.762523 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:25.762538 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:25.763262 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:25.763368 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:25.763397 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:25.763513 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:25.763759 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:27:25.764063 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":333,"threads_started":1}
I20260806 22:27:25.767333 9764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:25.768666 9764 log.cc:824] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:25.769194 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:27:25.771378 9764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5: No bootstrap required, opened a new log
I20260806 22:27:25.778460 9764 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } }
I20260806 22:27:25.778802 9764 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:25.778908 9764 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf7eb2290cf403a9d41a8deb0f75ab5, State: Initialized, Role: FOLLOWER
I20260806 22:27:25.779428 9764 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } }
I20260806 22:27:25.779718 9764 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:25.779867 9764 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:25.780040 9764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:25.782361 9764 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } }
I20260806 22:27:25.782845 9764 leader_election.cc:302] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf7eb2290cf403a9d41a8deb0f75ab5; no voters:
I20260806 22:27:25.783406 9764 leader_election.cc:288] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:25.783548 9772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:25.784291 9772 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [term 1 LEADER]: Becoming Leader. State: Replica: dcf7eb2290cf403a9d41a8deb0f75ab5, State: Running, Role: LEADER
I20260806 22:27:25.784868 9772 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } }
I20260806 22:27:25.785537 9764 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:25.787690 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } } }
I20260806 22:27:25.788480 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:25.787957 9775 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf7eb2290cf403a9d41a8deb0f75ab5. Latest consensus state: current_term: 1 leader_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf7eb2290cf403a9d41a8deb0f75ab5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40055 } } }
I20260806 22:27:25.788632 9775 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:25.792814 9781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:25.798004 9781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:25.806921 9781 catalog_manager.cc:1347] Generated new cluster ID: 88505a2819ac470f9fac0510d84ad24b
I20260806 22:27:25.807057 9781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:25.825050 9781 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:25.826295 9781 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:25.841338 9781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf7eb2290cf403a9d41a8deb0f75ab5: Generated new TSK 0
I20260806 22:27:25.842211 9781 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:27:25.872018 9804 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:27:25.872248 9804 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:27:25.873176 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:25.853657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.1.8.126:40055)
0806 22:27:25.853872 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:25.853880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:25.854383 (+ 503us) server_negotiation.cc:407] Connection header received
0806 22:27:25.854616 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:25.854620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:25.854684 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:25.854809 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:27:25.855363 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.856087 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.858090 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.858492 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.858586 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:25.859127 (+ 541us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:27:25.872678 (+ 13551us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:27:25.872955 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 22:27:25.873317 9768 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:27:25.873876 9803 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:25.853633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40055 (local address 127.0.0.1:53998)
0806 22:27:25.854273 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:25.854299 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:25.854440 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:25.854849 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:25.854854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:25.854867 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:27:25.855215 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:25.855229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.856297 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.856302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:25.857953 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:25.857969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:25.858603 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:25.858652 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:25.858861 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:25.858911 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:27:25.872889 (+ 13978us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:27:25.873174 (+ 285us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.8.126:40055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":348,"threads_started":1}
I20260806 22:27:25.875465 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 9652
I20260806 22:27:25.876580 9758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:25.971246 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 9652
2026-08-06T22:27:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (762 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:27:26.037720 1057 mini_oidc.cc:152] Starting JWKS server
I20260806 22:27:26.039867 1057 webserver.cc:460] Webserver started at https://127.0.0.1:46499/ using document root <none> and password file <none>
I20260806 22:27:26.040129 1057 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:27:26.040771 1057 webserver.cc:460] Webserver started at http://127.0.0.1:40957/ using document root <none> and password file <none>
2026-08-06T22:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:26Z Disabled control of system clock
I20260806 22:27:26.076597 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:41027
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:34843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:41027
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46499/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:27:26.415577 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:26.415912 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:26.416049 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:26.423213 9816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:26.423394 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:26.423483 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:26.423584 9816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:26.423669 9816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:27:26.423756 9816 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:27:26.423831 9816 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46499/jwks.json
I20260806 22:27:26.433223 9816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:41027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46499/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:26.434592 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:26.436601 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:26.449221 9827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:26.449178 9830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:26.449254 9826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:26.450132 9816 server_base.cc:1034] running on GCE node
I20260806 22:27:26.450954 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:26.452425 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:26.453655 9816 hybrid_clock.cc:648] HybridClock initialized: now 1786055246453617 us; error 57 us; skew 500 ppm
I20260806 22:27:26.454007 9816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:26.456296 9816 webserver.cc:460] Webserver started at http://127.1.8.126:34441/ using document root <none> and password file <none>
I20260806 22:27:26.457005 9816 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:26.457139 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:26.457491 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:26.460515 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "085c91f3659b460b832c8977814ea01a"
format_stamp: "Formatted at 2026-08-06 22:27:26 on dist-test-slave-7vjv"
server_key: "038533b7234268119cf37f9f65fa40ec"
server_key_iv: "8e564d24bd5d330687cf40a1ccfd387f"
server_key_version: "encryptionkey@0"
I20260806 22:27:26.461324 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "085c91f3659b460b832c8977814ea01a"
format_stamp: "Formatted at 2026-08-06 22:27:26 on dist-test-slave-7vjv"
server_key: "038533b7234268119cf37f9f65fa40ec"
server_key_iv: "8e564d24bd5d330687cf40a1ccfd387f"
server_key_version: "encryptionkey@0"
I20260806 22:27:26.466506 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:26.469944 9836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:26.471220 9816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:26.471427 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "085c91f3659b460b832c8977814ea01a"
format_stamp: "Formatted at 2026-08-06 22:27:26 on dist-test-slave-7vjv"
server_key: "038533b7234268119cf37f9f65fa40ec"
server_key_iv: "8e564d24bd5d330687cf40a1ccfd387f"
server_key_version: "encryptionkey@0"
I20260806 22:27:26.471653 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:26.492060 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:26.492973 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:26.493275 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:26.512712 9816 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:41027
I20260806 22:27:26.512753 9934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:41027 every 8 connection(s)
I20260806 22:27:26.514070 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:26.517917 9937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:26.518079 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 9816
I20260806 22:27:26.518280 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:26.519001 1057 external_mini_cluster.cc:1428] Setting key 29af199d0968423bb6d955b54fd06ac6
I20260806 22:27:26.531750 9937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a: Bootstrap starting.
I20260806 22:27:26.533355 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:26.520894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.1.8.126:41027)
0806 22:27:26.521932 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:26.521943 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:26.521984 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:27:26.523969 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:26.523991 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:26.524531 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:26.524873 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:26.525491 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.526739 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.528677 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.529379 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.529513 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:26.530768 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:26.530775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:26.530788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:26.531303 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:26.531380 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:26.531399 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:26.531484 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:26.531733 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:27:26.532309 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":388,"threads_started":1}
I20260806 22:27:26.535449 9937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:26.536755 9937 log.cc:824] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:26.537684 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:27:26.539656 9937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a: No bootstrap required, opened a new log
I20260806 22:27:26.546871 9937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } }
I20260806 22:27:26.547248 9937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:26.547359 9937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085c91f3659b460b832c8977814ea01a, State: Initialized, Role: FOLLOWER
I20260806 22:27:26.547909 9937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } }
I20260806 22:27:26.548213 9937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:26.548367 9937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:26.548547 9937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:26.550769 9937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } }
I20260806 22:27:26.551296 9937 leader_election.cc:302] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 085c91f3659b460b832c8977814ea01a; no voters:
I20260806 22:27:26.552026 9937 leader_election.cc:288] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:26.552140 9944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:26.553014 9944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [term 1 LEADER]: Becoming Leader. State: Replica: 085c91f3659b460b832c8977814ea01a, State: Running, Role: LEADER
I20260806 22:27:26.553558 9944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } }
I20260806 22:27:26.554251 9937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:26.556545 9946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "085c91f3659b460b832c8977814ea01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } } }
I20260806 22:27:26.557015 9946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:26.557648 9947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 085c91f3659b460b832c8977814ea01a. Latest consensus state: current_term: 1 leader_uuid: "085c91f3659b460b832c8977814ea01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085c91f3659b460b832c8977814ea01a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 41027 } } }
I20260806 22:27:26.558112 9947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:26.559294 9957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:26.564328 9957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:26.573809 9957 catalog_manager.cc:1347] Generated new cluster ID: 4d007b62ca9542bb895134cb99da8721
I20260806 22:27:26.573949 9957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:26.595633 9957 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:26.596973 9957 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:26.612526 9957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 085c91f3659b460b832c8977814ea01a: Generated new TSK 0
I20260806 22:27:26.613698 9957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:27:26.640309 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:26.630739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.1.8.126:41027)
0806 22:27:26.630967 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:26.630974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:26.631395 (+ 421us) server_negotiation.cc:407] Connection header received
0806 22:27:26.631645 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:26.631651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:26.631763 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:26.631947 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:27:26.632559 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.633719 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.635433 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.636191 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.636335 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:26.636967 (+ 632us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:27:26.639542 (+ 2575us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:27:26.639916 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 22:27:26.640457 9940 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:27:26.642189 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:26.630547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:41027 (local address 127.0.0.1:42112)
0806 22:27:26.631289 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:26.631307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:26.631443 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:26.632001 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:26.632005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:26.632017 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:27:26.632381 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:26.632392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.633905 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.633910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:26.635267 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:26.635286 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:26.636355 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:26.636435 (+ 80us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:26.636590 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:26.636632 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:27:26.639796 (+ 3164us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:27:26.641734 (+ 1938us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.8.126:41027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":1598,"thread_start_us":414,"threads_started":1}
I20260806 22:27:26.643484 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 9816
I20260806 22:27:26.644858 9929 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 22:27:26.656812 9978 webserver.cc:533] Webserver: sslize failed:
E20260806 22:27:26.656965 9978 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:27:26.769908 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 9816
2026-08-06T22:27:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (797 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:26Z Disabled control of system clock
I20260806 22:27:26.871417 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37203
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:37203
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/testchainca.cert with env {}
W20260806 22:27:27.211589 9984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:27.211886 9984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:27.211977 9984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:27.219303 9984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:27.219424 9984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:27.219462 9984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:27.219512 9984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:27.219549 9984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:27:27.219635 9984 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:27:27.219676 9984 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:27:27.230839 9984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:37203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:27.232182 9984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:27.233597 9984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:27.245716 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:27.245831 9992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:27.245806 9994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:27.246630 9984 server_base.cc:1034] running on GCE node
I20260806 22:27:27.247408 9984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:27.248868 9984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:27.250064 9984 hybrid_clock.cc:648] HybridClock initialized: now 1786055247250007 us; error 46 us; skew 500 ppm
I20260806 22:27:27.250416 9984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:27:27.252689 9984 webserver.cc:460] Webserver started at http://127.1.8.126:34831/ using document root <none> and password file <none>
I20260806 22:27:27.253417 9984 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:27.253548 9984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:27.253878 9984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:27.256808 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "55abad8068c44b15b0997f6fa20115a6"
format_stamp: "Formatted at 2026-08-06 22:27:27 on dist-test-slave-7vjv"
server_key: "f0bce17d21f3c0405f45dcb2b5bbceb1"
server_key_iv: "b7acfb116d5ed01fbf1ecfef84de5615"
server_key_version: "encryptionkey@0"
I20260806 22:27:27.257637 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "55abad8068c44b15b0997f6fa20115a6"
format_stamp: "Formatted at 2026-08-06 22:27:27 on dist-test-slave-7vjv"
server_key: "f0bce17d21f3c0405f45dcb2b5bbceb1"
server_key_iv: "b7acfb116d5ed01fbf1ecfef84de5615"
server_key_version: "encryptionkey@0"
I20260806 22:27:27.262665 9984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:27.266083 10005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:27.267689 9984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:27.267853 9984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "55abad8068c44b15b0997f6fa20115a6"
format_stamp: "Formatted at 2026-08-06 22:27:27 on dist-test-slave-7vjv"
server_key: "f0bce17d21f3c0405f45dcb2b5bbceb1"
server_key_iv: "b7acfb116d5ed01fbf1ecfef84de5615"
server_key_version: "encryptionkey@0"
I20260806 22:27:27.268028 9984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:27.290860 9984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:27.291697 9984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:27.291949 9984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:27.310513 9984 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:37203
I20260806 22:27:27.310531 10082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:37203 every 8 connection(s)
I20260806 22:27:27.311964 9984 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:27.314750 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 9984
I20260806 22:27:27.314977 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:27.315829 1057 external_mini_cluster.cc:1428] Setting key da96cb570bd9ea6a756ff6989f91e49b
I20260806 22:27:27.316398 10084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:27.329919 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:27.317803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.1.8.126:37203)
0806 22:27:27.318756 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:27.318767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:27.318811 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:27:27.320994 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:27.321016 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:27.321501 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:27.321811 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:27.322424 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.323805 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.325578 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.326235 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.326373 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:27.327630 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:27.327637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:27.327650 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:27:27.328279 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:27.328365 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:27:27.328386 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:27.328473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:27.328697 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 22:27:27.329023 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":575,"thread_start_us":369,"threads_started":1}
I20260806 22:27:27.330190 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6: Bootstrap starting.
I20260806 22:27:27.333632 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:27:27.333626 10084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:27.334817 10084 log.cc:824] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:27.337677 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6: No bootstrap required, opened a new log
I20260806 22:27:27.344650 10084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } }
I20260806 22:27:27.344987 10084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:27.345140 10084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55abad8068c44b15b0997f6fa20115a6, State: Initialized, Role: FOLLOWER
I20260806 22:27:27.345759 10084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } }
I20260806 22:27:27.346015 10084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:27.346168 10084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:27.346343 10084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:27.348554 10084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } }
I20260806 22:27:27.349073 10084 leader_election.cc:302] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55abad8068c44b15b0997f6fa20115a6; no voters:
I20260806 22:27:27.349684 10084 leader_election.cc:288] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:27.349869 10092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:27.350565 10092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [term 1 LEADER]: Becoming Leader. State: Replica: 55abad8068c44b15b0997f6fa20115a6, State: Running, Role: LEADER
I20260806 22:27:27.351189 10092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } }
I20260806 22:27:27.352026 10084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:27.354172 10094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55abad8068c44b15b0997f6fa20115a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } } }
I20260806 22:27:27.354182 10095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55abad8068c44b15b0997f6fa20115a6. Latest consensus state: current_term: 1 leader_uuid: "55abad8068c44b15b0997f6fa20115a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55abad8068c44b15b0997f6fa20115a6" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37203 } } }
I20260806 22:27:27.354641 10094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:27.354641 10095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:27.356462 10100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:27.361258 10100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:27.370505 10100 catalog_manager.cc:1347] Generated new cluster ID: a62834e7e94a4889910c6f263a3119cf
I20260806 22:27:27.370632 10100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:27.392576 10100 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:27.394368 10100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:27.405206 10100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55abad8068c44b15b0997f6fa20115a6: Generated new TSK 0
I20260806 22:27:27.406252 10100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:27:27.535326 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:27.420737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.1.8.126:37203)
0806 22:27:27.420970 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:27.420977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:27.421372 (+ 395us) server_negotiation.cc:407] Connection header received
0806 22:27:27.421583 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:27.421586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:27.421658 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:27.421781 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:27:27.422335 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.423030 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.425057 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.425507 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.425598 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:27.426199 (+ 601us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:27:27.534527 (+108328us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:27:27.534986 (+ 459us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60056: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 22:27:27.535477 10089 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60056: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:27:27.535477 10128 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:27.420606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:37203 (local address 127.0.0.1:60056)
0806 22:27:27.421261 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:27.421282 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:27.421432 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:27.421835 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:27.421840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:27.421854 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:27:27.422185 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:27.422198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.423173 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.423178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:27.424918 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:27.424934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:27.425631 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:27.425688 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:27.425893 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:27.425946 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:27:27.534856 (+108910us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:27:27.535122 (+ 266us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.8.126:37203: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":324,"threads_started":1}
I20260806 22:27:27.537055 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 9984
I20260806 22:27:27.538285 10076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:27.644220 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 9984
2026-08-06T22:27:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (853 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (424 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (435 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:28 dist-test-slave-7vjv krb5kdc[10155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:28 dist-test-slave-7vjv krb5kdc[10155](info): set up 2 sockets
Aug 06 22:27:28 dist-test-slave-7vjv krb5kdc[10155](info): commencing operation
krb5kdc: starting...
W20260806 22:27:29.860199 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:30 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:30.103020 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40237
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:30.441782 10170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:30.442072 10170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:30.442162 10170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:30.449285 10170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:30.449397 10170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:30.449445 10170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:30.449504 10170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:30.449550 10170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:30.459221 10170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:30.460639 10170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:30.461907 10170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:30.473868 10176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:30.473852 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:30.473974 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:30.475121 10170 server_base.cc:1034] running on GCE node
I20260806 22:27:30.475911 10170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:30.477334 10170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:30.478565 10170 hybrid_clock.cc:648] HybridClock initialized: now 1786055250478520 us; error 50 us; skew 500 ppm
I20260806 22:27:30.478904 10170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:30 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055250, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:30.482196 10170 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:30.483717 10170 webserver.cc:460] Webserver started at http://127.1.8.126:38739/ using document root <none> and password file <none>
I20260806 22:27:30.484447 10170 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:30.484551 10170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:30.484799 10170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:30.487646 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "d0707b5697af4b138b071975c6d7d6c0"
format_stamp: "Formatted at 2026-08-06 22:27:30 on dist-test-slave-7vjv"
server_key: "67dca30720e27fdb000ae44513b9eb53"
server_key_iv: "936ee149c3e86c868b37f410104208e6"
server_key_version: "encryptionkey@0"
I20260806 22:27:30.488451 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "d0707b5697af4b138b071975c6d7d6c0"
format_stamp: "Formatted at 2026-08-06 22:27:30 on dist-test-slave-7vjv"
server_key: "67dca30720e27fdb000ae44513b9eb53"
server_key_iv: "936ee149c3e86c868b37f410104208e6"
server_key_version: "encryptionkey@0"
I20260806 22:27:30.493539 10170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 22:27:30.496963 10189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:30.498229 10170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:30.498404 10170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "d0707b5697af4b138b071975c6d7d6c0"
format_stamp: "Formatted at 2026-08-06 22:27:30 on dist-test-slave-7vjv"
server_key: "67dca30720e27fdb000ae44513b9eb53"
server_key_iv: "936ee149c3e86c868b37f410104208e6"
server_key_version: "encryptionkey@0"
I20260806 22:27:30.498600 10170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:30.517900 10170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:30.522364 10170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:30.522658 10170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:30.541275 10170 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40237
I20260806 22:27:30.541316 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40237 every 8 connection(s)
I20260806 22:27:30.542601 10170 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:30.546411 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 10170
I20260806 22:27:30.546499 10273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:30.546665 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:30.547523 1057 external_mini_cluster.cc:1428] Setting key 4df6892d0ac855f12a20ce6f3993c179
I20260806 22:27:30.560917 10273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0: Bootstrap starting.
Aug 06 22:27:30 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:30.564759 10273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:30.565864 10273 log.cc:824] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:30.568889 10273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0: No bootstrap required, opened a new log
I20260806 22:27:30.571959 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:30.549724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.1.8.126:40237)
0806 22:27:30.550577 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:30.550591 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:30.550632 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:27:30.553009 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:30.553032 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:30.553433 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:30.553761 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:30.555478 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:30.556838 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:30.558699 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:30.559361 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:30.559493 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:30.564393 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:30.564402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:30.564417 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:30.564455 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:30.567647 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:30.568489 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:30.568496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:30.568503 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:30.568672 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:30.569106 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:30.569109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:30.569111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:30.569671 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:30.569988 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:30.570453 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 22:27:30.570909 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":426,"threads_started":1}
I20260806 22:27:30.576972 10273 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } }
I20260806 22:27:30.577409 10273 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:30.577525 10273 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0707b5697af4b138b071975c6d7d6c0, State: Initialized, Role: FOLLOWER
I20260806 22:27:30.578231 10273 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } }
I20260806 22:27:30.578514 10273 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:30.578682 10273 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:30.578877 10273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:30.581198 10273 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } }
I20260806 22:27:30.581770 10273 leader_election.cc:302] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0707b5697af4b138b071975c6d7d6c0; no voters:
I20260806 22:27:30.582355 10273 leader_election.cc:288] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:30.582497 10279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:30.584025 10279 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [term 1 LEADER]: Becoming Leader. State: Replica: d0707b5697af4b138b071975c6d7d6c0, State: Running, Role: LEADER
I20260806 22:27:30.584690 10273 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:30.584558 10279 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } }
I20260806 22:27:30.588032 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0707b5697af4b138b071975c6d7d6c0. Latest consensus state: current_term: 1 leader_uuid: "d0707b5697af4b138b071975c6d7d6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } } }
I20260806 22:27:30.588212 10280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0707b5697af4b138b071975c6d7d6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0707b5697af4b138b071975c6d7d6c0" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40237 } } }
I20260806 22:27:30.588649 10280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:30.588644 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:30.589525 10291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:30.595395 10291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:30.604641 10291 catalog_manager.cc:1347] Generated new cluster ID: 8e4e08ed519b471c8eabe1209e0fa5c9
I20260806 22:27:30.604775 10291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:30.631093 10291 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:30.632567 10291 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:30.645674 10291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0707b5697af4b138b071975c6d7d6c0: Generated new TSK 0
I20260806 22:27:30.646510 10291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:30.723282 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:31.064801 10308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:31.065085 10308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:31.065253 10308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:31.072774 10308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:31.072878 10308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:31.073088 10308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:31.082777 10308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:31.084126 10308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:31.085477 10308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:31.099020 10314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:31.099041 10315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:31.099159 10317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:31.101505 10308 server_base.cc:1034] running on GCE node
I20260806 22:27:31.102212 10308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:31.103153 10308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:31.104394 10308 hybrid_clock.cc:648] HybridClock initialized: now 1786055251104362 us; error 31 us; skew 500 ppm
I20260806 22:27:31.104723 10308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:31 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:31.111954 10308 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:31.113605 10308 webserver.cc:460] Webserver started at http://127.1.8.65:37159/ using document root <none> and password file <none>
I20260806 22:27:31.114368 10308 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:31.114521 10308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:31.114854 10308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:31.118026 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "dab5ac585f0d4f2e983ed9a2b00cdebc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "e375063a3ee0dd14fce91627bbe42418"
server_key_iv: "54e970be0f1aecf9e432853aefd13bf9"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.118863 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "dab5ac585f0d4f2e983ed9a2b00cdebc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "e375063a3ee0dd14fce91627bbe42418"
server_key_iv: "54e970be0f1aecf9e432853aefd13bf9"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.124495 10308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 22:27:31.127892 10327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.129213 10308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:31.129392 10308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "dab5ac585f0d4f2e983ed9a2b00cdebc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "e375063a3ee0dd14fce91627bbe42418"
server_key_iv: "54e970be0f1aecf9e432853aefd13bf9"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.129609 10308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:31.153708 10308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:31.157759 10308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:31.158080 10308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:31.159035 10308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:31.160411 10308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:31.160516 10308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.160678 10308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:31.160753 10308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.195128 10308 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:35705
I20260806 22:27:31.195176 10502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:35705 every 8 connection(s)
I20260806 22:27:31.196570 10308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:31.205438 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 10308
I20260806 22:27:31.205705 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:31.206354 1057 external_mini_cluster.cc:1428] Setting key c95f2c1014caf73ed6c33c0d91ce0e32
Aug 06 22:27:31 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:31.217142 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:31.199196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:51473 (local address 127.1.8.126:40237)
0806 22:27:31.199701 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:31.199705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:31.200608 (+ 903us) server_negotiation.cc:407] Connection header received
0806 22:27:31.202538 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:31.202543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:31.202642 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:31.202804 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:31.204368 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.205297 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.207065 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.207522 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.207665 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:31.211246 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:31.211251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:31.211255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:31.211287 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:31.213911 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:31.214677 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:31.214681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:31.214684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:31.214801 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:31.215267 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:31.215272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:31.215277 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:31.215550 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:31.215733 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:31.216517 (+ 784us) server_negotiation.cc:299] Negotiation successful
0806 22:27:31.216752 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":275,"threads_started":1}
I20260806 22:27:31.218134 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:31.199659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40237 (local address 127.1.8.65:51473)
0806 22:27:31.200438 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:31.200476 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:31.202272 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:31.203019 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:31.203027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:31.203531 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:31.204183 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:31.204203 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.205460 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.205464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:31.206872 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:31.206907 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.207748 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.207753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:31.207963 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:31.208792 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:31.208815 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:31.211014 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:31.214090 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:31.214100 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:31.214117 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:31.214536 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:31.214921 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:31.214926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:31.214930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:31.215143 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:31.215708 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:31.215716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:31.216058 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 22:27:31.216397 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 22:27:31.216949 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":292,"threads_started":1}
I20260806 22:27:31.219825 10503 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40237
I20260806 22:27:31.220212 10503 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:31.220983 10503 heartbeater.cc:511] Master 127.1.8.126:40237 requested a full tablet report, sending...
I20260806 22:27:31.223163 10213 ts_manager.cc:194] Registered new tserver with Master: dab5ac585f0d4f2e983ed9a2b00cdebc (127.1.8.65:35705)
I20260806 22:27:31.224731 10213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:51473
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:27:31.267599 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:31.596683 10512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:31.596968 10512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:31.597074 10512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:31.604476 10512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:31.604588 10512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:31.604722 10512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:31.614297 10512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:31.615507 10512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:31.616791 10512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:31.630272 10518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:31.630254 10519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:31.630612 10521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:31.631232 10512 server_base.cc:1034] running on GCE node
I20260806 22:27:31.631770 10512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:31.632682 10512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:31.633875 10512 hybrid_clock.cc:648] HybridClock initialized: now 1786055251633817 us; error 50 us; skew 500 ppm
I20260806 22:27:31.634258 10512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:31 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:31.637331 10512 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:31.638764 10512 webserver.cc:460] Webserver started at http://127.1.8.66:42599/ using document root <none> and password file <none>
I20260806 22:27:31.639446 10512 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:31.639608 10512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:31.639923 10512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:31.642941 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "b5a0d01349384c528e312387f8cdf1bc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "69a3847b595c7a8ad11f7cc026b579a4"
server_key_iv: "efeccebf3c4107eb816eb26ad763842a"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.643728 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "b5a0d01349384c528e312387f8cdf1bc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "69a3847b595c7a8ad11f7cc026b579a4"
server_key_iv: "efeccebf3c4107eb816eb26ad763842a"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.648693 10512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:27:31.652011 10534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.653304 10512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:31.653479 10512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "b5a0d01349384c528e312387f8cdf1bc"
format_stamp: "Formatted at 2026-08-06 22:27:31 on dist-test-slave-7vjv"
server_key: "69a3847b595c7a8ad11f7cc026b579a4"
server_key_iv: "efeccebf3c4107eb816eb26ad763842a"
server_key_version: "encryptionkey@0"
I20260806 22:27:31.653693 10512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:31.666218 10512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:31.669948 10512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:31.670248 10512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:31.671159 10512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:31.672502 10512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:31.672607 10512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.672758 10512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:31.672837 10512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:31.706565 10512 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:37885
I20260806 22:27:31.706601 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:37885 every 8 connection(s)
I20260806 22:27:31.707842 10512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:31.711839 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 10512
I20260806 22:27:31.712049 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:31.712656 1057 external_mini_cluster.cc:1428] Setting key 4389ae51737650a0fb3556ea0c9f538e
Aug 06 22:27:31 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:31.728144 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:31.710517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:38147 (local address 127.1.8.126:40237)
0806 22:27:31.710732 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:31.710738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:31.712118 (+ 1380us) server_negotiation.cc:407] Connection header received
0806 22:27:31.714384 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:31.714389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:31.714489 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:31.714591 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:31.716047 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.716812 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.718322 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.718732 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.718862 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:31.722321 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:31.722327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:31.722330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:31.722360 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:31.725003 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:31.725764 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:31.725773 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:31.725778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:31.725996 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:31.726442 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:31.726448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:31.726451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:31.726726 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:31.726887 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:31.727546 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 22:27:31.727774 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:27:31.729064 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:31.711041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40237 (local address 127.1.8.66:38147)
0806 22:27:31.711922 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:31.711973 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:31.714129 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:31.714833 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:31.714841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:31.715320 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:31.715882 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:31.715898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.716996 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.717000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:31.718158 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:31.718171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:31.718972 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:31.718980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:31.719201 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:31.719908 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:31.719930 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:31.722102 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:31.725159 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:31.725167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:31.725181 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:31.725618 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:31.726122 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:31.726129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:31.726132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:31.726327 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:31.726859 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:31.726867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:31.727192 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 22:27:31.727444 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 22:27:31.727955 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":339,"threads_started":1}
I20260806 22:27:31.730487 10721 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40237
I20260806 22:27:31.730854 10721 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:31.731721 10721 heartbeater.cc:511] Master 127.1.8.126:40237 requested a full tablet report, sending...
I20260806 22:27:31.733379 10213 ts_manager.cc:194] Registered new tserver with Master: b5a0d01349384c528e312387f8cdf1bc (127.1.8.66:37885)
I20260806 22:27:31.734683 10213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:38147
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:27:31.771334 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:32.105458 10728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:32.105806 10728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:32.105980 10728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:32.113157 10728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:32.113312 10728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:32.113502 10728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:32.123106 10728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:32.124670 10728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:32.126040 10728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:32.139631 10740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:32.139652 10748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:32.139559 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:32.140316 10728 server_base.cc:1034] running on GCE node
I20260806 22:27:32.140961 10728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:32.141830 10728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:32.143071 10728 hybrid_clock.cc:648] HybridClock initialized: now 1786055252143019 us; error 39 us; skew 500 ppm
I20260806 22:27:32.143383 10728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.146735 10728 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:32.148159 10728 webserver.cc:460] Webserver started at http://127.1.8.67:43787/ using document root <none> and password file <none>
I20260806 22:27:32.148834 10728 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:32.148936 10728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:32.149271 10728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:32.152302 10728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "ed4113a9ea26405296d170b5c6e1c6f2"
format_stamp: "Formatted at 2026-08-06 22:27:32 on dist-test-slave-7vjv"
server_key: "be8f4dccc3a3c639804f87c65e9f0b6f"
server_key_iv: "5ebf36df751918dd93396b26486cd234"
server_key_version: "encryptionkey@0"
I20260806 22:27:32.153025 10728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "ed4113a9ea26405296d170b5c6e1c6f2"
format_stamp: "Formatted at 2026-08-06 22:27:32 on dist-test-slave-7vjv"
server_key: "be8f4dccc3a3c639804f87c65e9f0b6f"
server_key_iv: "5ebf36df751918dd93396b26486cd234"
server_key_version: "encryptionkey@0"
I20260806 22:27:32.158653 10728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:32.162074 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:32.163353 10728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:32.163494 10728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "ed4113a9ea26405296d170b5c6e1c6f2"
format_stamp: "Formatted at 2026-08-06 22:27:32 on dist-test-slave-7vjv"
server_key: "be8f4dccc3a3c639804f87c65e9f0b6f"
server_key_iv: "5ebf36df751918dd93396b26486cd234"
server_key_version: "encryptionkey@0"
I20260806 22:27:32.163671 10728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:32.179059 10728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:32.183555 10728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:32.183821 10728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:32.184937 10728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:32.186391 10728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:32.186470 10728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:32.186558 10728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:32.186609 10728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:32.222127 10728 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:44883
I20260806 22:27:32.222165 10937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:44883 every 8 connection(s)
I20260806 22:27:32.223491 10728 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:32.225826 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 10728
I20260806 22:27:32.226030 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:32.226591 1057 external_mini_cluster.cc:1428] Setting key 94a567e6e989ec13aa65adec74b52145
I20260806 22:27:32.227867 10503 heartbeater.cc:503] Master 127.1.8.126:40237 was elected leader, sending a full tablet report...
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:32.247581 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.227022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40237 (local address 127.1.8.67:40285)
0806 22:27:32.228625 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:32.228684 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:32.230666 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:32.231272 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:32.231280 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:32.231785 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:32.232421 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:32.232439 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.233589 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.233592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:32.234842 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:32.234855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.235583 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.235587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:32.235786 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.236449 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:32.236471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:32.238620 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:32.242626 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:32.242635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:32.242655 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:32.243311 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:32.243723 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:32.243728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:32.243732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:32.243935 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:32.244562 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:32.244571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:32.244926 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 22:27:32.245249 (+ 323us) client_negotiation.cc:240] Negotiation successful
0806 22:27:32.245823 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1281,"thread_start_us":1074,"threads_started":1}
I20260806 22:27:32.248788 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.226540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:40285 (local address 127.1.8.126:40237)
0806 22:27:32.226786 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.226792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.228783 (+ 1991us) server_negotiation.cc:407] Connection header received
0806 22:27:32.230906 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.230910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.230975 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.231057 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.232585 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.233430 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.234982 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.235390 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.235488 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.238814 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.238820 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.238823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.238853 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.242492 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.243443 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.243449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.243452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.243604 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.244060 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.244065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.244068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.244426 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.244597 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.248184 (+ 3587us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.248441 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:27:32.254947 10940 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40237
I20260806 22:27:32.255328 10940 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:32.256600 10940 heartbeater.cc:511] Master 127.1.8.126:40237 requested a full tablet report, sending...
I20260806 22:27:32.259094 10211 ts_manager.cc:194] Registered new tserver with Master: ed4113a9ea26405296d170b5c6e1c6f2 (127.1.8.67:44883)
I20260806 22:27:32.260545 10211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:40285
I20260806 22:27:32.268801 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/test-admin.keytab.
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.304231 1057 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:27:32.309733 10950 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:32.307152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41452 (local address 127.1.8.65:35705)
0806 22:27:32.307805 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.307812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.307838 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:27:32.307904 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.307911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.308204 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.308377 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.309317 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41452: BlockingRecv error: recv got EOF from 127.0.0.1:41452 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":315,"threads_started":1}
W20260806 22:27:32.309784 1057 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.311667 1057 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.313046 1057 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:32.327499 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.314576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.1.8.65:35705)
0806 22:27:32.314787 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.314793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.314989 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:27:32.315122 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.315126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.315180 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.315257 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.316257 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.317206 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.318346 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.318854 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.318949 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.322431 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.322442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.322455 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.322493 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.325097 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.325688 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.325694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.325696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.325817 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.326125 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.326129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.326130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.326449 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.326661 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.326939 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.327215 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 22:27:32.333387 10950 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:32.331403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.1.8.65:35705)
0806 22:27:32.331643 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.331649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.331946 (+ 297us) server_negotiation.cc:407] Connection header received
0806 22:27:32.332040 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.332044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.332136 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.332219 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.333092 (+ 873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41470: BlockingRecv error: recv got EOF from 127.0.0.1:41470 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 22:27:32.333771 1057 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc)
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.335973 1057 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.337806 1057 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:32.352751 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.339635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.1.8.65:35705)
0806 22:27:32.339842 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.339848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.340189 (+ 341us) server_negotiation.cc:407] Connection header received
0806 22:27:32.340340 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.340348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.340428 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.340530 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.341903 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.342575 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.343745 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.344228 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.344334 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.347744 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.347749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.347752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.347814 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.350459 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.351032 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.351035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.351038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.351154 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.351463 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.351466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.351468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.351726 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.351925 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.352248 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.352430 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 22:27:32.356897 10950 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:32.355355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.1.8.65:35705)
0806 22:27:32.355572 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.355578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.355688 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:27:32.355807 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.355811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.355875 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.355945 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.356633 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41496: BlockingRecv error: recv got EOF from 127.0.0.1:41496 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 22:27:32.357100 1057 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055191034056-1057-0/krb5kdc/krb5cc)
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.358683 1057 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:32.359951 1057 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:32.373456 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.361226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.1.8.65:35705)
0806 22:27:32.361428 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.361436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.361699 (+ 263us) server_negotiation.cc:407] Connection header received
0806 22:27:32.361873 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.361882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.361934 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.362031 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.362964 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.363689 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.365003 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.365414 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.365544 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.368994 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.368999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.369002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.369031 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.371322 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.371882 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.371885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.371888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.372027 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.372399 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.372402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.372404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.372645 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.372875 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.373047 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.373208 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:27:32.375121 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 10308
I20260806 22:27:32.387179 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.375826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.1.8.65:35705)
0806 22:27:32.375995 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.376003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.376348 (+ 345us) server_negotiation.cc:407] Connection header received
0806 22:27:32.376528 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.376537 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.376589 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.376686 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.377607 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.378264 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.379474 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.379904 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.380039 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.381988 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.381993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.381996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.382024 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.384410 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.385190 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.385197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.385200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.385338 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.385704 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.385756 (+ 52us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.385759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.386037 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.386318 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.386621 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.386832 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:27:32.387812 10493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:32.472275 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 10308
I20260806 22:27:32.500566 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 10512
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:27:32.516582 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.501539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.1.8.66:37885)
0806 22:27:32.502369 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.502376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.502403 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:27:32.502465 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.502472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.502710 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.502885 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.504014 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.505113 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.506417 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.506992 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.507137 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.511444 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.511455 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.511468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.511500 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.514185 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.514784 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.514791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.514793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.514906 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.515225 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.515229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.515230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.515541 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.515787 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.516029 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.516291 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":433,"threads_started":1}
I20260806 22:27:32.517828 10708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:32.600427 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 10512
I20260806 22:27:32.625440 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 10728
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:27:32.640733 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:32.626466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.1.8.67:44883)
0806 22:27:32.627142 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:32.627149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:32.627181 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:27:32.627247 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:32.627254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:32.627462 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:32.627606 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:32.628698 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.629724 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.630950 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:32.631507 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:32.631612 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:32.635421 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:32.635432 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:32.635447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:32.635479 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:32.638209 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.638814 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.638821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.638823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.638949 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:32.639273 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:32.639276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:32.639278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:32.639641 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:32.639880 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:32.640144 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 22:27:32.640407 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":369,"threads_started":1}
I20260806 22:27:32.642040 10930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:32.724156 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 10728
I20260806 22:27:32.751358 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 10170
I20260806 22:27:32.752678 10262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:32.859251 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 10170
2026-08-06T22:27:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4355 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10977](info): set up 2 sockets
Aug 06 22:27:32 dist-test-slave-7vjv krb5kdc[10977](info): commencing operation
krb5kdc: starting...
W20260806 22:27:34.178516 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:34 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab.
Entry for principal oryx/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab.
I20260806 22:27:34.453801 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33507
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab
--principal=oryx/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:34.795037 10999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:34.795313 10999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:34.795481 10999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:34.802778 10999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:34.802908 10999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:34.802999 10999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:34.803100 10999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:34.803179 10999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:34.812914 10999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab
--principal=oryx/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:34.814230 10999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:34.815572 10999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:34.827219 11010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:34.827342 11011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:34.827282 11016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:34.828454 10999 server_base.cc:1034] running on GCE node
I20260806 22:27:34.829313 10999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:34.830683 10999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:34.831888 10999 hybrid_clock.cc:648] HybridClock initialized: now 1786055254831839 us; error 45 us; skew 500 ppm
I20260806 22:27:34.832301 10999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:34 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:34.835361 10999 init.cc:377] Logged in from keytab as oryx/127.1.8.126@KRBTEST.COM (short username oryx)
I20260806 22:27:34.836899 10999 webserver.cc:460] Webserver started at http://127.1.8.126:35889/ using document root <none> and password file <none>
I20260806 22:27:34.837586 10999 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:34.837719 10999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:34.838047 10999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:34.841148 10999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "a7f980e5a1294880bcc4bb32f47bbe79"
format_stamp: "Formatted at 2026-08-06 22:27:34 on dist-test-slave-7vjv"
server_key: "b2a94148b26d646e619ea5d55c8c599f"
server_key_iv: "89443eabbbc3b7283353cbcbcba5b8a0"
server_key_version: "encryptionkey@0"
I20260806 22:27:34.841988 10999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "a7f980e5a1294880bcc4bb32f47bbe79"
format_stamp: "Formatted at 2026-08-06 22:27:34 on dist-test-slave-7vjv"
server_key: "b2a94148b26d646e619ea5d55c8c599f"
server_key_iv: "89443eabbbc3b7283353cbcbcba5b8a0"
server_key_version: "encryptionkey@0"
I20260806 22:27:34.847393 10999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:27:34.850848 11026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:34.852202 10999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:27:34.852375 10999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "a7f980e5a1294880bcc4bb32f47bbe79"
format_stamp: "Formatted at 2026-08-06 22:27:34 on dist-test-slave-7vjv"
server_key: "b2a94148b26d646e619ea5d55c8c599f"
server_key_iv: "89443eabbbc3b7283353cbcbcba5b8a0"
server_key_version: "encryptionkey@0"
I20260806 22:27:34.852579 10999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:34.863507 10999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:34.867416 10999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:34.867702 10999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:34.886595 10999 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:33507
I20260806 22:27:34.886647 11103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:33507 every 8 connection(s)
I20260806 22:27:34.887989 10999 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:34.890079 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 10999
I20260806 22:27:34.890326 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:34.891191 1057 external_mini_cluster.cc:1428] Setting key 98836b6298474e444bb48fff76a673b5
I20260806 22:27:34.892787 11104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:27:34 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:34.907927 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Bootstrap starting.
I20260806 22:27:34.911464 11104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:34.912758 11104 log.cc:824] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:34.915386 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:34.893372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.1.8.126:33507)
0806 22:27:34.894148 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:34.894159 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:34.894199 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:27:34.895677 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:34.895696 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:34.896079 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:34.896473 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:34.898332 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:34.899690 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:34.901558 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:34.902306 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:34.902497 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:34.907522 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:34.907530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:34.907546 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:34.907593 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:34.910778 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:34.911663 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:34.911672 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:34.911680 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:34.911907 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:34.912353 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:34.912357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:34.912359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:34.912987 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:34.913228 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:34.913882 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 22:27:34.914367 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":353,"threads_started":1}
I20260806 22:27:34.916077 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: No bootstrap required, opened a new log
I20260806 22:27:34.924235 11104 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } }
I20260806 22:27:34.924670 11104 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:34.924784 11104 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f980e5a1294880bcc4bb32f47bbe79, State: Initialized, Role: FOLLOWER
I20260806 22:27:34.925387 11104 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } }
I20260806 22:27:34.925657 11104 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:34.925810 11104 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:34.926008 11104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:34.928679 11104 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } }
I20260806 22:27:34.929229 11104 leader_election.cc:302] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7f980e5a1294880bcc4bb32f47bbe79; no voters:
I20260806 22:27:34.929838 11104 leader_election.cc:288] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:34.929983 11111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:34.930799 11111 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 LEADER]: Becoming Leader. State: Replica: a7f980e5a1294880bcc4bb32f47bbe79, State: Running, Role: LEADER
I20260806 22:27:34.931461 11111 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } }
I20260806 22:27:34.932207 11104 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:34.935413 11115 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f980e5a1294880bcc4bb32f47bbe79. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } }
I20260806 22:27:34.935887 11115 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:34.936272 11114 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } }
I20260806 22:27:34.936691 11114 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:34.941109 11121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:34.946386 11121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:34.955758 11121 catalog_manager.cc:1347] Generated new cluster ID: 69a518de17674a02819485d85bb3de2f
I20260806 22:27:34.955885 11121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:34.972404 11121 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:34.973953 11121 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:34.981781 11121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Generated new TSK 0
I20260806 22:27:34.982587 11121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
I20260806 22:27:35.047461 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:35.390342 11143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:35.390616 11143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:35.390709 11143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:35.397919 11143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:35.398028 11143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:35.398186 11143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:35.408174 11143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:35.409531 11143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:35.410859 11143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:35.424964 11157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:35.425107 11152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:35.425010 11153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:35.425626 11143 server_base.cc:1034] running on GCE node
I20260806 22:27:35.426262 11143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:35.427126 11143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:35.428315 11143 hybrid_clock.cc:648] HybridClock initialized: now 1786055255428255 us; error 50 us; skew 500 ppm
I20260806 22:27:35.428625 11143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:35 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055255, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:35.431689 11143 init.cc:377] Logged in from keytab as oryx/127.1.8.65@KRBTEST.COM (short username oryx)
I20260806 22:27:35.433199 11143 webserver.cc:460] Webserver started at http://127.1.8.65:46361/ using document root <none> and password file <none>
I20260806 22:27:35.433964 11143 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:35.434064 11143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:35.434315 11143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:35.437182 11143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "4c99f6382fd44e2e92cf342c70c22db9"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "d46c62a5fb23a5dbd191a1833a36d6f3"
server_key_iv: "391d0a2d5a379e41c4ee6ffce1e6b661"
server_key_version: "encryptionkey@0"
I20260806 22:27:35.437937 11143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "4c99f6382fd44e2e92cf342c70c22db9"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "d46c62a5fb23a5dbd191a1833a36d6f3"
server_key_iv: "391d0a2d5a379e41c4ee6ffce1e6b661"
server_key_version: "encryptionkey@0"
I20260806 22:27:35.442889 11143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:27:35.446441 11164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:35.447731 11143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 22:27:35.447911 11143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "4c99f6382fd44e2e92cf342c70c22db9"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "d46c62a5fb23a5dbd191a1833a36d6f3"
server_key_iv: "391d0a2d5a379e41c4ee6ffce1e6b661"
server_key_version: "encryptionkey@0"
I20260806 22:27:35.448184 11143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:35.494886 11143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:35.498957 11143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:35.499277 11143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:35.500350 11143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:35.501730 11143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:35.501837 11143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:35.501992 11143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:35.502068 11143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:35.536168 11143 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:40025
I20260806 22:27:35.536187 11353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:40025 every 8 connection(s)
I20260806 22:27:35.537488 11143 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:35.545089 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 11143
I20260806 22:27:35.545360 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:35.546032 1057 external_mini_cluster.cc:1428] Setting key fe46488fd1098ff1fbbb8ba9101cfcd9
Aug 06 22:27:35 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055255, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:35.559125 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:35.540086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60197 (local address 127.1.8.126:33507)
0806 22:27:35.540561 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:35.540566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:35.541531 (+ 965us) server_negotiation.cc:407] Connection header received
0806 22:27:35.543337 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:35.543342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:35.543435 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:35.543557 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:35.545183 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:35.545956 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:35.547753 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:35.548306 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:35.548417 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:35.552985 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:35.552991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:35.552995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:35.553025 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:35.555807 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:35.556796 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:35.556799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:35.556802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:35.556928 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:35.557419 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:35.557422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:35.557424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:35.557698 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:35.557810 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:35.558704 (+ 894us) server_negotiation.cc:299] Negotiation successful
0806 22:27:35.558870 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":224,"threads_started":1}
I20260806 22:27:35.560665 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:35.540704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:60197)
0806 22:27:35.541373 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:35.541411 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:35.543073 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:35.543818 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:35.543826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:35.544413 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:35.545013 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:35.545031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:35.546122 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:35.546149 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:35.547519 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:35.547532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:35.548547 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:35.548556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:35.548853 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:35.549617 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:35.549658 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:35.552742 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:35.555926 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:35.555938 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:35.555957 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:35.556657 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:35.557037 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:35.557044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:35.557051 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:35.557301 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:35.557825 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:35.557835 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:35.558301 (+ 466us) client_negotiation.cc:769] Sending connection context
0806 22:27:35.558610 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 22:27:35.559195 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260806 22:27:35.562377 11355 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:35.562680 11355 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:35.563391 11355 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:35.565729 11053 ts_manager.cc:194] Registered new tserver with Master: 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025)
I20260806 22:27:35.567998 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:60197
WARNING: no policy specified for oryx/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
I20260806 22:27:35.607452 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:35.949074 11363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:35.949347 11363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:35.949530 11363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:35.956894 11363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:35.957024 11363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:35.957223 11363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:35.966815 11363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:35.968217 11363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:35.969592 11363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:35.982921 11372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:35.982868 11377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:35.982957 11375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:35.983773 11363 server_base.cc:1034] running on GCE node
I20260806 22:27:35.984488 11363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:35.985430 11363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:35.986675 11363 hybrid_clock.cc:648] HybridClock initialized: now 1786055255986639 us; error 45 us; skew 500 ppm
I20260806 22:27:35.987002 11363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:35 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055255, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:35.990227 11363 init.cc:377] Logged in from keytab as oryx/127.1.8.66@KRBTEST.COM (short username oryx)
I20260806 22:27:35.991712 11363 webserver.cc:460] Webserver started at http://127.1.8.66:34835/ using document root <none> and password file <none>
I20260806 22:27:35.992493 11363 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:35.992651 11363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:35.992981 11363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:35.995918 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "90de8ee9d5054e64941641985d35adea"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "e9060eda7f7cc4acd6f33907be1b2df1"
server_key_iv: "5a72cc5c732817c1a6d9712932689604"
server_key_version: "encryptionkey@0"
I20260806 22:27:35.996757 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "90de8ee9d5054e64941641985d35adea"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "e9060eda7f7cc4acd6f33907be1b2df1"
server_key_iv: "5a72cc5c732817c1a6d9712932689604"
server_key_version: "encryptionkey@0"
I20260806 22:27:36.001865 11363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:36.005532 11387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.006870 11363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:36.007046 11363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "90de8ee9d5054e64941641985d35adea"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "e9060eda7f7cc4acd6f33907be1b2df1"
server_key_iv: "5a72cc5c732817c1a6d9712932689604"
server_key_version: "encryptionkey@0"
I20260806 22:27:36.007251 11363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:36.020637 11363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:36.024567 11363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:36.024864 11363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:36.025785 11363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:36.027089 11363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:36.027199 11363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.027357 11363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:36.027433 11363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.061545 11363 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39213
I20260806 22:27:36.061589 11568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39213 every 8 connection(s)
I20260806 22:27:36.062942 11363 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:36.064956 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 11363
I20260806 22:27:36.065193 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:36.065814 1057 external_mini_cluster.cc:1428] Setting key c32c24f05556ee86fcd9132d943107db
Aug 06 22:27:36 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055255, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:36.084300 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:36.065585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:56943 (local address 127.1.8.126:33507)
0806 22:27:36.066081 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:36.066087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:36.067315 (+ 1228us) server_negotiation.cc:407] Connection header received
0806 22:27:36.069349 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:36.069358 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:36.069452 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:36.069578 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:36.071034 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.072214 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.073675 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.074389 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.074534 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:36.077999 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:36.078008 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:36.078015 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:36.078059 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:36.080995 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:36.081937 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:36.081940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:36.081942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:36.082067 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:36.082546 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:36.082549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:36.082551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:36.082809 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:36.082933 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:36.083786 (+ 853us) server_negotiation.cc:299] Negotiation successful
0806 22:27:36.083951 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"spinlock_wait_cycles":768,"thread_start_us":317,"threads_started":1}
I20260806 22:27:36.085813 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:36.066261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:56943)
0806 22:27:36.067133 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:36.067181 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:36.069072 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:36.069764 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:36.069771 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:36.070286 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:36.070848 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:36.070864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.072357 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.072361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:36.073528 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:36.073540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.074610 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.074614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:36.074811 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:36.075503 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:36.075526 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:36.077780 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:36.081146 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:36.081159 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:36.081177 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:36.081820 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:36.082178 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:36.082185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:36.082192 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:36.082436 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:36.082950 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:36.082960 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:36.083405 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 22:27:36.083721 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 22:27:36.084320 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":344,"threads_started":1}
I20260806 22:27:36.087393 11569 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:36.087708 11569 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:36.088510 11569 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:36.090108 11053 ts_manager.cc:194] Registered new tserver with Master: 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:36.091444 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:56943
WARNING: no policy specified for oryx/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
I20260806 22:27:36.128818 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:36.476611 11578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:36.476905 11578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:36.477078 11578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:36.484352 11578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:36.484462 11578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:36.484653 11578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:36.494714 11578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:36.496119 11578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:36.497493 11578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:36.511317 11587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:36.511353 11588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:36.511407 11593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:36.512290 11578 server_base.cc:1034] running on GCE node
I20260806 22:27:36.512940 11578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:36.513947 11578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:36.515163 11578 hybrid_clock.cc:648] HybridClock initialized: now 1786055256515119 us; error 44 us; skew 500 ppm
I20260806 22:27:36.515501 11578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:36 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055256, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:36.522622 11578 init.cc:377] Logged in from keytab as oryx/127.1.8.67@KRBTEST.COM (short username oryx)
I20260806 22:27:36.524509 11578 webserver.cc:460] Webserver started at http://127.1.8.67:33153/ using document root <none> and password file <none>
I20260806 22:27:36.525245 11578 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:36.525350 11578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:36.525627 11578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:36.528678 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "49b3ef451a98414e95aca376cddd79ab"
format_stamp: "Formatted at 2026-08-06 22:27:36 on dist-test-slave-7vjv"
server_key: "4ccba93dbc602733c6dfa527ae6dbfef"
server_key_iv: "47eb010d4ab313eb358cb73103230e9c"
server_key_version: "encryptionkey@0"
I20260806 22:27:36.529425 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "49b3ef451a98414e95aca376cddd79ab"
format_stamp: "Formatted at 2026-08-06 22:27:36 on dist-test-slave-7vjv"
server_key: "4ccba93dbc602733c6dfa527ae6dbfef"
server_key_iv: "47eb010d4ab313eb358cb73103230e9c"
server_key_version: "encryptionkey@0"
I20260806 22:27:36.534802 11578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 22:27:36.539247 11603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.540886 11578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:36.541079 11578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "49b3ef451a98414e95aca376cddd79ab"
format_stamp: "Formatted at 2026-08-06 22:27:36 on dist-test-slave-7vjv"
server_key: "4ccba93dbc602733c6dfa527ae6dbfef"
server_key_iv: "47eb010d4ab313eb358cb73103230e9c"
server_key_version: "encryptionkey@0"
I20260806 22:27:36.541311 11578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:36.564440 11578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:36.569201 11578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:36.569530 11578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:36.570607 11578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:36.571085 11355 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
I20260806 22:27:36.572407 11578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:36.572504 11578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.572638 11578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:36.572702 11578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:36.608143 11578 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:43835
I20260806 22:27:36.608258 11787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:43835 every 8 connection(s)
I20260806 22:27:36.609464 11578 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:36.617038 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 11578
I20260806 22:27:36.617259 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:36.617897 1057 external_mini_cluster.cc:1428] Setting key 66e18317964a0d19ecf58f0d844795c5
Aug 06 22:27:36 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055256, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:36.630702 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:36.612240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:38395 (local address 127.1.8.126:33507)
0806 22:27:36.612772 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:36.612777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:36.613715 (+ 938us) server_negotiation.cc:407] Connection header received
0806 22:27:36.615608 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:36.615614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:36.615706 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:36.615833 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:36.617854 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.618833 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.620700 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.621199 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.621331 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:36.624809 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:36.624817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:36.624825 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:36.624876 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:36.627765 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:36.628578 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:36.628582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:36.628585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:36.628730 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:36.629178 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:36.629182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:36.629185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:36.629471 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:36.629608 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:36.630247 (+ 639us) server_negotiation.cc:299] Negotiation successful
0806 22:27:36.630428 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":227,"threads_started":1}
I20260806 22:27:36.631660 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:36.612788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:38395)
0806 22:27:36.613523 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:36.613570 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:36.615326 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:36.616138 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:36.616150 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:36.616800 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:36.617663 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:36.617687 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.618995 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.619001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:36.620550 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:36.620562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:36.621428 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:36.621437 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:36.621686 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:36.622299 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:36.622327 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:36.624593 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:36.627898 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:36.627906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:36.627918 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:36.628427 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:36.628845 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:36.628848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:36.628850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:36.629042 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:36.629583 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:36.629589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:36.629901 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 22:27:36.630142 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 22:27:36.630520 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":296,"threads_started":1}
I20260806 22:27:36.633294 11789 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:36.633656 11789 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:36.634404 11789 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:36.635912 11053 ts_manager.cc:194] Registered new tserver with Master: 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
I20260806 22:27:36.636783 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:38395
I20260806 22:27:36.645931 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.125.keytab.
Entry for principal oryx/127.1.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.125.keytab.
I20260806 22:27:36.709877 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:43001
--webserver_interface=127.1.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.125.keytab
--principal=oryx/127.1.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33507,127.1.8.125:43001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:37.094245 11569 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
W20260806 22:27:37.102511 11796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:37.102854 11796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:37.103024 11796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:37.110839 11796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:37.110994 11796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:37.111110 11796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:37.111203 11796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:37.111253 11796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:37.122116 11796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33507,127.1.8.125:43001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.125.keytab
--principal=oryx/127.1.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:43001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
W20260806 22:27:37.122434 11796 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.8.126:33507,127.1.8.125:43001'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:27:37.123497 11796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:37.125125 11796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:37.137910 11806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:37.137945 11805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:37.137795 11808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:37.139323 11796 server_base.cc:1034] running on GCE node
I20260806 22:27:37.140265 11796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:37.141706 11796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:37.142966 11796 hybrid_clock.cc:648] HybridClock initialized: now 1786055257142904 us; error 49 us; skew 500 ppm
I20260806 22:27:37.143323 11796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:37.146930 11796 init.cc:377] Logged in from keytab as oryx/127.1.8.125@KRBTEST.COM (short username oryx)
I20260806 22:27:37.148623 11796 webserver.cc:460] Webserver started at http://127.1.8.125:43327/ using document root <none> and password file <none>
I20260806 22:27:37.149343 11796 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:37.149478 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:37.149850 11796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:37.153340 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/instance:
uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
format_stamp: "Formatted at 2026-08-06 22:27:37 on dist-test-slave-7vjv"
server_key: "1e6a094cd9dc1cccd984090757cf38af"
server_key_iv: "adf0410048eec4e1936f73e2637bd9d8"
server_key_version: "encryptionkey@0"
I20260806 22:27:37.154189 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal/instance:
uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
format_stamp: "Formatted at 2026-08-06 22:27:37 on dist-test-slave-7vjv"
server_key: "1e6a094cd9dc1cccd984090757cf38af"
server_key_iv: "adf0410048eec4e1936f73e2637bd9d8"
server_key_version: "encryptionkey@0"
I20260806 22:27:37.159812 11796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 22:27:37.163384 11819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:37.164803 11796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 22:27:37.164984 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
format_stamp: "Formatted at 2026-08-06 22:27:37 on dist-test-slave-7vjv"
server_key: "1e6a094cd9dc1cccd984090757cf38af"
server_key_iv: "adf0410048eec4e1936f73e2637bd9d8"
server_key_version: "encryptionkey@0"
I20260806 22:27:37.165212 11796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:37.182264 11796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:37.186791 11796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:37.187094 11796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:37.209323 11796 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.125:43001
I20260806 22:27:37.209373 11897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.125:43001 every 8 connection(s)
I20260806 22:27:37.210856 11796 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
I20260806 22:27:37.212788 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 11796
I20260806 22:27:37.215934 11898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:37.224879 11898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.125@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:37.239866 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.215217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.1.8.125:43001)
0806 22:27:37.216151 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.216172 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.216223 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:27:37.218000 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.218022 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.218418 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.218783 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.220891 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.222531 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.224387 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.225130 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.225259 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.230804 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.230815 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.230828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.230887 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.234910 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.235825 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.235831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.235837 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.236044 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.236548 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.236555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.236562 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.237344 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.237677 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.238195 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.238660 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":378,"threads_started":1}
I20260806 22:27:37.246363 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.228379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.1.8.126:33507)
0806 22:27:37.229881 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.229888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.229913 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:27:37.229966 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.229970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.230047 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.230200 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.231898 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.232929 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.235206 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.235908 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.236009 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.239903 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.239909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.239912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.240001 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.243107 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.243979 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.243983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.243985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.244170 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.244599 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.244604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.244607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.245106 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.245249 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.245721 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.245972 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1383,"thread_start_us":1225,"threads_started":1}
I20260806 22:27:37.246460 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.228278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42404)
0806 22:27:37.229262 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.229304 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.229507 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.230262 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.230270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.231266 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.231704 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.231716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.233074 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.233087 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.235030 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.235043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.236150 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.236160 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.236327 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.236947 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.236968 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.239697 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.243249 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.243260 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.243267 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.243852 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.244291 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.244296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.244299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.244493 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.245238 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.245245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.245415 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.245626 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.245972 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":526,"threads_started":1}
I20260806 22:27:37.248448 11898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.125@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
I20260806 22:27:37.261103 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.249248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50224)
0806 22:27:37.249441 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.249459 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.249546 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.249796 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.249798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.250145 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.250281 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.250289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.251279 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.251282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.252316 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.252327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.253004 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.253008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.253152 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.253716 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.253732 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.255772 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.258847 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.258850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.258852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.259266 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.259590 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.259593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.259606 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.259733 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.260338 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.260342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.260498 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.260624 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.260839 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:27:37.261338 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.249264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.1.8.125:43001)
0806 22:27:37.249454 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.249460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.249520 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:27:37.249633 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.249636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.249688 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.249762 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.250393 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.251153 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.252447 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.252857 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.252959 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.255974 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.255978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.255981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.256008 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.258727 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.259364 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.259367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.259369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.259495 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.259822 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.259825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.259827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.260205 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.260317 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.260714 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.260980 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":81}
I20260806 22:27:37.266590 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Bootstrap starting.
I20260806 22:27:37.270210 11898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:37.271315 11898 log.cc:824] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:37.274933 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: No bootstrap required, opened a new log
I20260806 22:27:37.282743 11898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } }
I20260806 22:27:37.283049 11898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:37.283134 11898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d35ca6da104f7eac8bbbcf3a37ef72, State: Initialized, Role: FOLLOWER
I20260806 22:27:37.283651 11898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } }
I20260806 22:27:37.284626 11909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } }
I20260806 22:27:37.285030 11909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:37.285671 11898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:27:37.295847 11923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:27:37.296073 11923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:27:37.298367 11796 master_runner.cc:428] Detected that this master 56d35ca6da104f7eac8bbbcf3a37ef72 is joining an existing cluster
I20260806 22:27:37.298489 11796 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.8.125:43001
I20260806 22:27:37.330989 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.303651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.1.8.125:43001)
0806 22:27:37.303904 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.303910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.312686 (+ 8776us) server_negotiation.cc:407] Connection header received
0806 22:27:37.312875 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.312881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.312945 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.313004 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.316682 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.318098 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.320269 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.321047 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.321157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.323737 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.323744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.323749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.323782 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.327790 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.328669 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.328674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.328677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.328833 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.329264 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.329268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.329272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.329646 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.329833 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.330312 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.330556 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:27:37.331630 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.303787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.1.8.126:33507)
0806 22:27:37.303993 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.303998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.311687 (+ 7689us) server_negotiation.cc:407] Connection header received
0806 22:27:37.312432 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.312439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.312616 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.312707 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.314730 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.315912 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.318396 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.319239 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.319364 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.322540 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.322549 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.322556 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.322631 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.326717 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.327851 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.327858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.327862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.328071 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.328746 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.328754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.328760 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.329271 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.329446 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.330287 (+ 841us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.330639 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:27:37.332042 11937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.304352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50232)
0806 22:27:37.312401 (+ 8049us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.312418 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.312766 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.313232 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.313238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.314057 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.316500 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.316519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.318330 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.318335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.320078 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.320122 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.321521 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.321531 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.321738 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.322383 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.322415 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.323545 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.327938 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.327943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.327948 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.328550 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.328965 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.328969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.328972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.329153 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.329909 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.329914 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.330059 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.331283 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.331610 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7914,"mutex_wait_us":1928,"thread_start_us":7358,"threads_started":1}
WARNING: no policy specified for oryx/127.1.8.124@KRBTEST.COM; defaulting to no policy
I20260806 22:27:37.333839 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.303739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42414)
0806 22:27:37.311349 (+ 7610us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.311373 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.312247 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.312947 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.312952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.313936 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.314549 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.314566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.316080 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.316085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.318216 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.318235 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.319738 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.319747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.320079 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.320866 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.320894 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.322331 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.326899 (+ 4568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.326907 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.326914 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.327693 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.328273 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.328285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.328291 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.328560 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.329556 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.329564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.329881 (+ 317us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.332871 (+ 2990us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.333322 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7441,"mutex_wait_us":81,"thread_start_us":8317,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.124@KRBTEST.COM" created.
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.126@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
I20260806 22:27:37.364167 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.342192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50238)
0806 22:27:37.344818 (+ 2626us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.344863 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.345147 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.345763 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.345772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.346819 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.347174 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.347190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.348734 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.348762 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.350438 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.350455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.351738 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.351746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.351990 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.352829 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.352853 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.356176 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.360546 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.360566 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.360577 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.361245 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.361740 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.361745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.361749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.361994 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.362760 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.362769 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.362994 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.363239 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.363611 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2306,"thread_start_us":2111,"threads_started":1}
I20260806 22:27:37.364358 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.342194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.1.8.125:43001)
0806 22:27:37.342382 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.342390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.344960 (+ 2570us) server_negotiation.cc:407] Connection header received
0806 22:27:37.345431 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.345439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.345554 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.345674 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.347345 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.348582 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.350619 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.351504 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.351720 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.356378 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.356384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.356389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.356438 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.360358 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.361398 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.361406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.361413 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.361602 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.362134 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.362143 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.362149 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.362577 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.362759 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.363417 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.363769 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:27:37.367022 11053 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.8.125:43001: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
member_type: NON_VOTER
last_known_addr {
host: "127.1.8.125"
port: 43001
}
attrs {
promote: true
}
}
dest_uuid: "a7f980e5a1294880bcc4bb32f47bbe79"
cas_config_opid_index: -1
I20260806 22:27:37.368731 11053 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } }
I20260806 22:27:37.373086 11946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.374202 11863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.124.keytab.
Entry for principal oryx/127.1.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.124.keytab.
I20260806 22:27:37.376457 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:33417
--webserver_interface=127.1.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.124.keytab
--principal=oryx/127.1.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:37.378322 11863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:27:37.380512 11948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:37.386126 11946 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.8.125:43001
I20260806 22:27:37.388388 11796 master.cc:537] Master@127.1.8.125:43001 shutting down...
I20260806 22:27:37.389356 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.389730 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:37.416831 11863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.422108 11961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.422662 11961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:37.423192 11909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f980e5a1294880bcc4bb32f47bbe79. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.423432 11909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:37.423499 11958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.423889 11958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:37.450834 11796 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 22:27:37.451408 11796 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 22:27:37.451608 11796 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: stopping tablet replica
I20260806 22:27:37.470589 11796 master.cc:552] Master@127.1.8.125:43001 shutdown complete.
I20260806 22:27:37.470786 11796 master_runner.cc:305] Clearing existing system tablet
I20260806 22:27:37.471293 11796 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:37.471410 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:37.474905 11796 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:27:37.480829 11965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:37.481292 11796 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:37.481606 11796 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:27:37.481702 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
format_stamp: "Formatted at 2026-08-06 22:27:37 on dist-test-slave-7vjv"
server_key: "1e6a094cd9dc1cccd984090757cf38af"
server_key_iv: "adf0410048eec4e1936f73e2637bd9d8"
server_key_version: "encryptionkey@0"
I20260806 22:27:37.486966 11796 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:37.495471 11796 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:27:37.495760 11796 log.cc:1196] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:27:37.496374 11796 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Deleting consensus metadata
I20260806 22:27:37.496675 11796 master_runner.cc:315] Copying system tablet from 127.1.8.126:33507
I20260806 22:27:37.499691 11796 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: tablet copy: Beginning tablet copy session from remote peer at address 127.1.8.126:33507
I20260806 22:27:37.518009 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.500850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.1.8.126:33507)
0806 22:27:37.501047 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.501053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.501708 (+ 655us) server_negotiation.cc:407] Connection header received
0806 22:27:37.501930 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.501936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.502047 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.502169 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.503335 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.504592 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.506596 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.507438 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.507575 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.510430 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.510437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.510442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.510491 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.514430 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.515385 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.515392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.515396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.515576 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.516088 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.516119 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.516122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.516553 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.516735 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.517268 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.517541 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:27:37.518066 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.500763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42420)
0806 22:27:37.501558 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.501580 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.501772 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.502230 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.502235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.502798 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.503144 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.503158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.504818 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.504825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.506413 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.506431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.507660 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.507669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.507913 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.508829 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.508865 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.510043 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.514627 (+ 4584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.514633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.514637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.515219 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.515719 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.515724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.515728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.515938 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.516715 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.516720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.516926 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.517123 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.517583 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":480,"threads_started":1}
I20260806 22:27:37.519491 11084 tablet_copy_service.cc:140] P a7f980e5a1294880bcc4bb32f47bbe79: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56d35ca6da104f7eac8bbbcf3a37ef72 ({username='oryx', principal='oryx/127.1.8.125@KRBTEST.COM'} at 127.0.0.1:42420)
I20260806 22:27:37.519841 11084 tablet_copy_service.cc:161] P a7f980e5a1294880bcc4bb32f47bbe79: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56d35ca6da104f7eac8bbbcf3a37ef72 at {username='oryx', principal='oryx/127.1.8.125@KRBTEST.COM'} at 127.0.0.1:42420: session id = 56d35ca6da104f7eac8bbbcf3a37ef72-00000000000000000000000000000000
I20260806 22:27:37.525190 11084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:27:37.527844 11796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:27:37.535071 11796 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: tablet copy: Starting download of 0 data blocks...
I20260806 22:27:37.535399 11796 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: tablet copy: Starting download of 1 WAL segments...
I20260806 22:27:37.538672 11796 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:27:37.544008 11084 tablet_copy_service.cc:342] P a7f980e5a1294880bcc4bb32f47bbe79: Request end of tablet copy session 56d35ca6da104f7eac8bbbcf3a37ef72-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.8.125@KRBTEST.COM'} at 127.0.0.1:42420
I20260806 22:27:37.544524 11084 tablet_copy_service.cc:433] P a7f980e5a1294880bcc4bb32f47bbe79: ending tablet copy session 56d35ca6da104f7eac8bbbcf3a37ef72-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56d35ca6da104f7eac8bbbcf3a37ef72
I20260806 22:27:37.557852 11796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:37.560403 11978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:37.565629 11980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:37.572381 11982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:37.573035 11796 server_base.cc:1034] running on GCE node
I20260806 22:27:37.573398 11796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:37.575588 11796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:37.576838 11796 hybrid_clock.cc:648] HybridClock initialized: now 1786055257576759 us; error 52 us; skew 500 ppm
I20260806 22:27:37.577073 11796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:37 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:37.579622 11796 init.cc:377] Logged in from keytab as oryx/127.1.8.125@KRBTEST.COM (short username oryx)
I20260806 22:27:37.585157 11796 webserver.cc:460] Webserver started at http://127.1.8.125:37723/ using document root <none> and password file <none>
I20260806 22:27:37.585541 11796 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:37.585644 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:37.589442 11796 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 22:27:37.591734 11992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:37.592283 11796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 22:27:37.592440 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
format_stamp: "Formatted at 2026-08-06 22:27:37 on dist-test-slave-7vjv"
server_key: "1e6a094cd9dc1cccd984090757cf38af"
server_key_iv: "adf0410048eec4e1936f73e2637bd9d8"
server_key_version: "encryptionkey@0"
I20260806 22:27:37.592633 11796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:37.614283 11796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:37.619485 11796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:37.639267 11789 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
I20260806 22:27:37.735790 11796 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.125:43001
I20260806 22:27:37.735832 12060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.125:43001 every 8 connection(s)
I20260806 22:27:37.737097 11796 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
I20260806 22:27:37.748492 12061 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:27:37.751688 12061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Bootstrap starting.
I20260806 22:27:37.766184 12061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:27:37.766891 12061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: Bootstrap complete.
I20260806 22:27:37.768553 12061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } }
I20260806 22:27:37.768920 12061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56d35ca6da104f7eac8bbbcf3a37ef72, State: Initialized, Role: LEARNER
I20260806 22:27:37.769310 12061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } }
I20260806 22:27:37.770638 12066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.771101 12061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:37.771377 12066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:37.776978 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.777458 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:37.809499 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.779066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.1.8.125:43001)
0806 22:27:37.781327 (+ 2261us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:37.781338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:37.781386 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:27:37.781498 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:37.781507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:37.781642 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:37.783696 (+ 2054us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:37.784281 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.787524 (+ 3243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.790257 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.791823 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.795567 (+ 3744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.796209 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:37.796221 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:37.796230 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:37.796281 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:37.803895 (+ 7614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.805431 (+ 1536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.805440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.805447 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.805736 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:37.806462 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:37.806471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:37.806477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:37.807251 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:37.808320 (+ 1069us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:37.808548 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 22:27:37.808880 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":971,"threads_started":1}
I20260806 22:27:37.809589 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:37.777459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50248)
0806 22:27:37.778100 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:37.778129 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:37.778306 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:37.781909 (+ 3603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:37.781918 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:37.782962 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:37.783459 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.783480 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.787759 (+ 4279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.787769 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.789988 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:37.790011 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:37.792071 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:37.792081 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:37.792330 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:37.793434 (+ 1104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:37.793473 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:37.795185 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:37.804156 (+ 8971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.804166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.804173 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.805118 (+ 945us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.805915 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:37.805924 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:37.805931 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:37.806224 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:37.807495 (+ 1271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:37.807502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:37.808063 (+ 561us) client_negotiation.cc:769] Sending connection context
0806 22:27:37.808288 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 22:27:37.808762 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395}
I20260806 22:27:37.812551 12066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f980e5a1294880bcc4bb32f47bbe79. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:37.813092 12066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:37.814929 11949 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: attempting to promote NON_VOTER 56d35ca6da104f7eac8bbbcf3a37ef72 to VOTER
I20260806 22:27:37.816330 11949 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } }
I20260806 22:27:37.834672 12030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 LEARNER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:27:37.837275 11948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:27:37.849151 11948 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } }
I20260806 22:27:37.870563 12030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } }
I20260806 22:27:37.873413 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } }
I20260806 22:27:37.873716 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:37.874234 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } }
I20260806 22:27:37.874550 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:37.878278 12066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } }
I20260806 22:27:37.878557 12066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:37.882356 12086 catalog_manager.cc:1259] Loaded cluster ID: 69a518de17674a02819485d85bb3de2f
I20260806 22:27:37.882537 12086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: loading cluster ID for follower catalog manager: success
I20260806 22:27:37.885910 12086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: acquiring CA information for follower catalog manager: success
I20260806 22:27:37.889753 12086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:27:38.024196 11950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:38.024739 11950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:38.024920 11950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:38.034996 11950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:38.035135 11950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:38.035195 11950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:38.035283 11950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:38.035331 11950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:38.047958 11950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.124.keytab
--principal=oryx/127.1.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:33417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:38.049485 11950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:38.050997 11950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:38.067854 12098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:38.068341 12099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:38.068972 12101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:38.070307 11950 server_base.cc:1034] running on GCE node
I20260806 22:27:38.071326 11950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:38.073105 11950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:38.074366 11950 hybrid_clock.cc:648] HybridClock initialized: now 1786055258074281 us; error 70 us; skew 500 ppm
I20260806 22:27:38.074764 11950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:38.079141 11950 init.cc:377] Logged in from keytab as oryx/127.1.8.124@KRBTEST.COM (short username oryx)
I20260806 22:27:38.081496 11950 webserver.cc:460] Webserver started at http://127.1.8.124:37697/ using document root <none> and password file <none>
I20260806 22:27:38.082415 11950 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:38.082620 11950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:38.083002 11950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:38.087138 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/instance:
uuid: "68d01b29806f487383336ffd1396148b"
format_stamp: "Formatted at 2026-08-06 22:27:38 on dist-test-slave-7vjv"
server_key: "220c0091677703d34a4dbe3d8eacb243"
server_key_iv: "4e82e71d72a7bc05596a1b445f1acbb5"
server_key_version: "encryptionkey@0"
I20260806 22:27:38.088431 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal/instance:
uuid: "68d01b29806f487383336ffd1396148b"
format_stamp: "Formatted at 2026-08-06 22:27:38 on dist-test-slave-7vjv"
server_key: "220c0091677703d34a4dbe3d8eacb243"
server_key_iv: "4e82e71d72a7bc05596a1b445f1acbb5"
server_key_version: "encryptionkey@0"
I20260806 22:27:38.095714 11950 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:27:38.101394 12111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:38.102846 11950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 22:27:38.103096 11950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "68d01b29806f487383336ffd1396148b"
format_stamp: "Formatted at 2026-08-06 22:27:38 on dist-test-slave-7vjv"
server_key: "220c0091677703d34a4dbe3d8eacb243"
server_key_iv: "4e82e71d72a7bc05596a1b445f1acbb5"
server_key_version: "encryptionkey@0"
I20260806 22:27:38.103375 11950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:38.134858 11950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:38.140739 11950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:38.141212 11950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:38.181406 11950 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.124:33417
I20260806 22:27:38.181586 12190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.124:33417 every 8 connection(s)
I20260806 22:27:38.183219 11950 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
I20260806 22:27:38.184187 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 11950
I20260806 22:27:38.189719 12194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:38.202145 12194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.124@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:38.222609 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.187363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.1.8.124:33417)
0806 22:27:38.188929 (+ 1566us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.188945 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.189284 (+ 339us) server_negotiation.cc:407] Connection header received
0806 22:27:38.193125 (+ 3841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.193151 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.193692 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.194199 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.196720 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.199620 (+ 2900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.201284 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.202510 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.202771 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.210840 (+ 8069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.210850 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.210865 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.210957 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.216348 (+ 5391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.217774 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.217783 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.217791 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.218122 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.218725 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.218731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.218734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.219857 (+ 1123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.220338 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.220838 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.221205 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1276,"thread_start_us":770,"threads_started":1}
I20260806 22:27:38.229873 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.205892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42432 (local address 127.1.8.126:33507)
0806 22:27:38.207133 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.207148 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.207911 (+ 763us) server_negotiation.cc:407] Connection header received
0806 22:27:38.208235 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.208240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.208333 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.208494 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.211907 (+ 3413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.212937 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.216052 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.217169 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.217331 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.222421 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.222425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.222428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.222464 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.227040 (+ 4576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.227867 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.227871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.227873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.228013 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.228420 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.228423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.228425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.228695 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.228812 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.229260 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.229415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":874,"threads_started":1}
I20260806 22:27:38.229883 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.206227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42432)
0806 22:27:38.207765 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.207802 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.208043 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.209660 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.209669 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.210809 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.211726 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.211742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.213198 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.213206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.215878 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.215908 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.217439 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.217444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.217632 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.218547 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.218563 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.222231 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.227159 (+ 4928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.227167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.227173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.227765 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.228158 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.228166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.228168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.228326 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.228807 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.228812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.228998 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.229191 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.229422 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"thread_start_us":1068,"threads_started":1}
I20260806 22:27:38.231326 12194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.124@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
I20260806 22:27:38.247747 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.232400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50260)
0806 22:27:38.232659 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.232677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.232810 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.233144 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.233148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.233885 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.234131 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.234140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.235222 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.235229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.236437 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.236449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.237174 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.237180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.237411 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.238046 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.238063 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.240828 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.244511 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.244516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.244520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.245209 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.245646 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.245650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.245654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.245857 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.246554 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.246559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.246729 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.246867 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.247306 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:27:38.247815 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.232410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.1.8.125:43001)
0806 22:27:38.232625 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.232631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.232754 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:27:38.232940 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.232945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.233018 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.233121 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.234277 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.235060 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.236577 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.237009 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.237137 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.241022 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.241035 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.241038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.241071 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.244360 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.245337 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.245343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.245348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.245529 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.245979 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.245990 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.245992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.246418 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.246588 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.246974 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.247295 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:27:38.249222 12194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.124@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
I20260806 22:27:38.265077 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.250182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.0.0.1:40256)
0806 22:27:38.250330 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.250345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.250429 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.250695 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.250698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.251273 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.251546 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.251555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.252804 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.252808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.253880 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.253891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.254663 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.254667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.254902 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.255607 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.255623 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.258982 (+ 3359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.262820 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.262823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.262826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.263156 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.263581 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.263585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.263587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.263727 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.264250 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.264253 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.264365 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.264481 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.264664 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:27:38.265172 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.250015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.1.8.124:33417)
0806 22:27:38.250180 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.250185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.250412 (+ 227us) server_negotiation.cc:407] Connection header received
0806 22:27:38.250518 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.250522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.250585 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.250660 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.251668 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.252679 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.254004 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.254509 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.254609 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.259171 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.259176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.259179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.259208 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.262707 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.263273 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.263277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.263279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.263486 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.263831 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.263834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.263836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.264147 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.264266 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.264560 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.264782 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":70}
I20260806 22:27:38.271039 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Bootstrap starting.
I20260806 22:27:38.275470 12194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:38.277637 12194 log.cc:824] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:38.282444 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: No bootstrap required, opened a new log
I20260806 22:27:38.292963 12194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } }
I20260806 22:27:38.293359 12194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:38.293654 12194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d01b29806f487383336ffd1396148b, State: Initialized, Role: FOLLOWER
I20260806 22:27:38.294337 12194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } }
I20260806 22:27:38.295775 12206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } } }
I20260806 22:27:38.296428 12206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:38.296994 12194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:27:38.312383 12221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:27:38.312631 12221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:27:38.318171 11950 master_runner.cc:428] Detected that this master 68d01b29806f487383336ffd1396148b is joining an existing cluster
I20260806 22:27:38.318338 11950 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.8.124:33417
I20260806 22:27:38.330996 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 11143
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.65@KRBTEST.COM
I20260806 22:27:38.363102 12230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.325959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.0.0.1:40260)
0806 22:27:38.330111 (+ 4152us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.330135 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.330330 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.330915 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.330964 (+ 49us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.334904 (+ 3940us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.335871 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.335904 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.338448 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.338454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.341207 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.341237 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.346237 (+ 5000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.346245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.346523 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.347665 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.347748 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.350171 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.357200 (+ 7029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.357207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.357212 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.357954 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.360356 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.360361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.360365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.360572 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.361598 (+ 1026us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.361603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.361819 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.362004 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.362375 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3951,"mutex_wait_us":22,"thread_start_us":2286,"threads_started":1}
I20260806 22:27:38.364495 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.325493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.1.8.124:33417)
0806 22:27:38.325960 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.325965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.330215 (+ 4250us) server_negotiation.cc:407] Connection header received
0806 22:27:38.330465 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.330470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.330591 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.330680 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.336195 (+ 5515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.338034 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.341433 (+ 3399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.342357 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.342498 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.350450 (+ 7952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.350457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.350461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.350494 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.356902 (+ 6408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.358123 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.358139 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.358145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.360221 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.360698 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.360703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.360706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.361290 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.361512 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.363525 (+ 2013us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.363830 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:27:38.369395 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.325293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.1.8.126:33507)
0806 22:27:38.325506 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.325511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.328767 (+ 3256us) server_negotiation.cc:407] Connection header received
0806 22:27:38.330223 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.330231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.330319 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.330398 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.337979 (+ 7581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.339785 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.350462 (+ 10677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.351416 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.351525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.358822 (+ 7297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.358830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.358834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.358888 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.364926 (+ 6038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.366014 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.366019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.366023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.366267 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.366905 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.366912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.366916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.367445 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.367610 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.368485 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.368810 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:27:38.371026 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.325229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42436)
0806 22:27:38.328625 (+ 3396us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.328651 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.329830 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.330721 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.330725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.335300 (+ 4575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.336297 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.336323 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.340111 (+ 3788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.340119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.347010 (+ 6891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.347025 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.351979 (+ 4954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.351988 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.352341 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.354409 (+ 2068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.354430 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.358479 (+ 4049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.365153 (+ 6674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.365160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.365163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.365855 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.366491 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.366497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.366502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.366701 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.367739 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.367744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.368121 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.369963 (+ 1842us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.370419 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3207,"mutex_wait_us":1,"thread_start_us":7764,"threads_started":1}
I20260806 22:27:38.377319 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.332833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.1.8.65:40025)
0806 22:27:38.334032 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.334040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.334080 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:27:38.334149 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.334157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.342760 (+ 8603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.345680 (+ 2920us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.346808 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.351558 (+ 4750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.354183 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.355413 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.364489 (+ 9076us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.365112 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.365132 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.365152 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.365226 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.370904 (+ 5678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.372019 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.372028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.372031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.372259 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.373294 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.373298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.373302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.373832 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.375816 (+ 1984us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.375990 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.376319 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":671,"threads_started":1}
I20260806 22:27:38.379016 11347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:38.385869 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.324976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.1.8.125:43001)
0806 22:27:38.325210 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.325216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.349909 (+ 24693us) server_negotiation.cc:407] Connection header received
0806 22:27:38.353112 (+ 3203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.353118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.353223 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.353300 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.358133 (+ 4833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.359671 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.370059 (+ 10388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.370936 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.371044 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.376633 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.376658 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.376665 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.376712 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.381250 (+ 4538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.382343 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.382352 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.382358 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.382639 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.383300 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.383308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.383313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.383899 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.384071 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.384899 (+ 828us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.385171 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:27:38.387228 12233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.326931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:50264)
0806 22:27:38.349777 (+ 22846us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.349816 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.352918 (+ 3102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.353494 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.353500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.357114 (+ 3614us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.357903 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.357918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.359909 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.359916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.369847 (+ 9931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.369881 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.371773 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.371784 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.372148 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.372992 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.373014 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.376321 (+ 3307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.381406 (+ 5085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.381414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.381421 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.382166 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.382829 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.382836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.382842 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.383148 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.384183 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.384191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.384572 (+ 381us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.386070 (+ 1498us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.386554 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22634,"thread_start_us":23004,"threads_started":1}
W20260806 22:27:38.394361 12121 tablet.cc:2369] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.126@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
I20260806 22:27:38.407310 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.379444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.0.0.1:40264)
0806 22:27:38.380551 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.380576 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.380784 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.383267 (+ 2483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.383272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.384137 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.384754 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.384793 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.386591 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.386599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.389836 (+ 3237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.389852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.391622 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.391629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.391926 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.392802 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.392826 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.397709 (+ 4883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.403987 (+ 6278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.403992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.403997 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.404689 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.405115 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.405120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.405124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.405325 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.406081 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.406085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.406226 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.406419 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.406698 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":811,"threads_started":1}
I20260806 22:27:38.408465 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.381527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.1.8.124:33417)
0806 22:27:38.381742 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.381748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.381765 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:27:38.381888 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.381895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.382061 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.382161 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.384957 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.386316 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.390001 (+ 3685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.391096 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.391253 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.397963 (+ 6710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.397971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.397976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.398015 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.403807 (+ 5792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.404821 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.404826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.404830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.404975 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.405443 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.405448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.405452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.405873 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.406004 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.407658 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.407917 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 22:27:38.414538 11053 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.8.124:33417: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "68d01b29806f487383336ffd1396148b"
member_type: NON_VOTER
last_known_addr {
host: "127.1.8.124"
port: 33417
}
attrs {
promote: true
}
}
dest_uuid: "a7f980e5a1294880bcc4bb32f47bbe79"
cas_config_opid_index: 6
I20260806 22:27:38.415386 11053 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } }
I20260806 22:27:38.418249 12030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:27:38.418432 12156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:38.419291 11948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:27:38.422971 11948 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 68d01b29806f487383336ffd1396148b (127.1.8.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.424297 12156 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:27:38.424978 12030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 68d01b29806f487383336ffd1396148b (127.1.8.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.440716 12240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.441210 12240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:38.441505 12237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.002s
I20260806 22:27:38.448705 11948 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.8.124:33417
I20260806 22:27:38.449303 11950 master.cc:537] Master@127.1.8.124:33417 shutting down...
I20260806 22:27:38.450055 12237 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.450532 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.450821 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:38.451242 12237 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:38.461609 12156 raft_consensus.cc:858] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } }
W20260806 22:27:38.461792 12156 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } }
I20260806 22:27:38.486265 12156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:38.493090 12250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:38.493729 12250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:27:38.493898 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.493966 12250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:38.494171 12250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:27:38.494180 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:38.494500 12206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f980e5a1294880bcc4bb32f47bbe79. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: NON_VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: true } } }
I20260806 22:27:38.494760 12206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260806 22:27:38.495806 11029 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 22:27:38.498184 11950 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
W20260806 22:27:38.498380 11029 consensus_peers.cc:487] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 -> Peer 68d01b29806f487383336ffd1396148b (127.1.8.124:33417): Couldn't send request to peer 68d01b29806f487383336ffd1396148b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:27:38.498692 11950 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 22:27:38.498869 11950 tablet_replica.cc:330] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: stopping tablet replica
I20260806 22:27:38.511803 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 11143
I20260806 22:27:38.519045 11950 master.cc:552] Master@127.1.8.124:33417 shutdown complete.
I20260806 22:27:38.519150 11950 master_runner.cc:305] Clearing existing system tablet
I20260806 22:27:38.519594 11950 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:38.519737 11950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:38.523006 11950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 22:27:38.524757 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:38.525125 11950 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:38.525434 11950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:27:38.525543 11950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "68d01b29806f487383336ffd1396148b"
format_stamp: "Formatted at 2026-08-06 22:27:38 on dist-test-slave-7vjv"
server_key: "220c0091677703d34a4dbe3d8eacb243"
server_key_iv: "4e82e71d72a7bc05596a1b445f1acbb5"
server_key_version: "encryptionkey@0"
I20260806 22:27:38.531246 11950 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:38.540278 11950 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:27:38.540479 11950 log.cc:1196] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:27:38.541074 11950 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Deleting consensus metadata
I20260806 22:27:38.541355 11950 master_runner.cc:315] Copying system tablet from 127.1.8.126:33507
I20260806 22:27:38.544808 11950 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: tablet copy: Beginning tablet copy session from remote peer at address 127.1.8.126:33507
I20260806 22:27:38.547217 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:40025
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:33507
--webserver_port=46361
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:38.562460 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.545781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:42442)
0806 22:27:38.546728 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:38.546751 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:38.546926 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:38.547413 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:38.547420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:38.548042 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:38.548392 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.548409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.549933 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.549941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.551664 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:38.551682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.552947 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.552953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:38.553149 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.553852 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:38.553883 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:38.555041 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:38.559442 (+ 4401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.559448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.559452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.559958 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.560421 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:38.560427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:38.560430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:38.560611 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:38.561303 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:38.561309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:38.561507 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:27:38.561686 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 22:27:38.562007 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":651,"threads_started":1}
I20260806 22:27:38.562580 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:38.545830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.1.8.126:33507)
0806 22:27:38.546240 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:38.546247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:38.546860 (+ 613us) server_negotiation.cc:407] Connection header received
0806 22:27:38.547090 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:38.547096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:38.547199 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:38.547347 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:38.548572 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.549751 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.551827 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:38.552580 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:38.552702 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:38.555260 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:38.555268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:38.555276 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:38.555320 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:38.559301 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.560127 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.560135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.560142 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.560304 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:38.560730 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:38.560738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:38.560744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:38.561159 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:38.561314 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:38.561818 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 22:27:38.562131 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260806 22:27:38.563547 11084 tablet_copy_service.cc:140] P a7f980e5a1294880bcc4bb32f47bbe79: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 68d01b29806f487383336ffd1396148b ({username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:42442)
I20260806 22:27:38.563768 11084 tablet_copy_service.cc:161] P a7f980e5a1294880bcc4bb32f47bbe79: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 68d01b29806f487383336ffd1396148b at {username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:42442: session id = 68d01b29806f487383336ffd1396148b-00000000000000000000000000000000
I20260806 22:27:38.568027 11084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:27:38.570046 11950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:27:38.577121 11950 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: tablet copy: Starting download of 0 data blocks...
I20260806 22:27:38.577497 11950 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: tablet copy: Starting download of 1 WAL segments...
I20260806 22:27:38.580179 11950 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:27:38.584630 11084 tablet_copy_service.cc:342] P a7f980e5a1294880bcc4bb32f47bbe79: Request end of tablet copy session 68d01b29806f487383336ffd1396148b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:42442
I20260806 22:27:38.584780 11084 tablet_copy_service.cc:433] P a7f980e5a1294880bcc4bb32f47bbe79: ending tablet copy session 68d01b29806f487383336ffd1396148b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 68d01b29806f487383336ffd1396148b
I20260806 22:27:38.597316 11950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:38.600181 12271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:38.600577 12272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:38.608477 11950 server_base.cc:1034] running on GCE node
W20260806 22:27:38.612632 12276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:38.613044 11950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:38.613679 11950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:38.614972 11950 hybrid_clock.cc:648] HybridClock initialized: now 1786055258614925 us; error 43 us; skew 500 ppm
I20260806 22:27:38.615200 11950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:38 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:38.617177 11950 init.cc:377] Logged in from keytab as oryx/127.1.8.124@KRBTEST.COM (short username oryx)
I20260806 22:27:38.624680 11950 webserver.cc:460] Webserver started at http://127.1.8.124:33921/ using document root <none> and password file <none>
I20260806 22:27:38.624980 11950 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:38.625097 11950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:38.627647 11950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:27:38.629480 12287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:38.629945 11950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:27:38.630082 11950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "68d01b29806f487383336ffd1396148b"
format_stamp: "Formatted at 2026-08-06 22:27:38 on dist-test-slave-7vjv"
server_key: "220c0091677703d34a4dbe3d8eacb243"
server_key_iv: "4e82e71d72a7bc05596a1b445f1acbb5"
server_key_version: "encryptionkey@0"
I20260806 22:27:38.630277 11950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:38.659494 11950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:38.663041 11950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:38.686169 11950 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.124:33417
I20260806 22:27:38.686205 12373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.124:33417 every 8 connection(s)
I20260806 22:27:38.687122 11950 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
I20260806 22:27:38.694419 12374 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:27:38.705925 12374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Bootstrap starting.
I20260806 22:27:38.748762 12374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:27:38.749267 12374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Bootstrap complete.
I20260806 22:27:38.750442 12374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } }
I20260806 22:27:38.750757 12374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 68d01b29806f487383336ffd1396148b, State: Initialized, Role: LEARNER
I20260806 22:27:38.750993 12374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } }
I20260806 22:27:38.765123 12374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:38.764812 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:38.765357 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:38.874914 12395 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:27:39.000635 12238 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:39.001106 12238 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:39.066470 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.001750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.0.0.1:40266)
0806 22:27:39.040005 (+ 38255us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:39.040036 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:39.040277 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:39.043051 (+ 2774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:39.043060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:39.044414 (+ 1354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:39.045347 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.045368 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.048450 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.048457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.050917 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.050939 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.052986 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.052996 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.053300 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.054841 (+ 1541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:39.054872 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:39.056640 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:39.062215 (+ 5575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.062220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.062224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.062903 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.063453 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.063458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.063462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.063703 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.064662 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:39.064669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:39.064856 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 22:27:39.065091 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 22:27:39.065758 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38076,"thread_start_us":40420,"threads_started":1}
I20260806 22:27:39.068353 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.002500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.1.8.124:33417)
0806 22:27:39.041499 (+ 38999us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:39.041509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:39.041536 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:27:39.041637 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:39.041646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:39.041768 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:39.041911 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:39.045661 (+ 3750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.048258 (+ 2597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.051366 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.052507 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.052672 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.057023 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:39.057032 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:39.057039 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:39.057091 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:39.062037 (+ 4946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.063070 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.063076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.063080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.063305 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.063837 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.063842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.063846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.064431 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:39.064580 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:39.067137 (+ 2557us) server_negotiation.cc:299] Negotiation successful
0806 22:27:39.067517 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38803,"thread_start_us":39910,"threads_started":1}
I20260806 22:27:39.070185 12333 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260806 22:27:39.070887 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f980e5a1294880bcc4bb32f47bbe79. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: NON_VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: true } } }
I20260806 22:27:39.071332 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:27:39.072080 12238 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: attempting to promote NON_VOTER 68d01b29806f487383336ffd1396148b to VOTER
I20260806 22:27:39.073062 12238 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } }
I20260806 22:27:39.076233 12333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 LEARNER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:27:39.076448 12030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Refusing update from remote peer a7f980e5a1294880bcc4bb32f47bbe79: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:27:39.077252 12238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:27:39.077622 12396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:27:39.081295 12030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68d01b29806f487383336ffd1396148b (127.1.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.083133 12407 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68d01b29806f487383336ffd1396148b (127.1.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.089905 12396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.089933 12238 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.090399 12396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:39.090579 12238 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:39.093897 12396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.094436 12396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:39.095196 12333 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68d01b29806f487383336ffd1396148b (127.1.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.104624 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.105075 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:39.105943 12411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:39.106374 12411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:27:39.178889 12270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:39.179208 12270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:39.179376 12270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:39.189502 12270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:39.189707 12270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:39.189993 12270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:39.203120 12270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:40025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=46361
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:39.204898 12270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:39.206493 12270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:39.224707 12420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:39.224761 12421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:39.224814 12424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:39.225944 12270 server_base.cc:1034] running on GCE node
I20260806 22:27:39.226796 12270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:39.227860 12270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:39.229102 12270 hybrid_clock.cc:648] HybridClock initialized: now 1786055259229034 us; error 47 us; skew 500 ppm
I20260806 22:27:39.229734 12270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:39.233757 12270 init.cc:377] Logged in from keytab as oryx/127.1.8.65@KRBTEST.COM (short username oryx)
I20260806 22:27:39.235426 12270 webserver.cc:460] Webserver started at http://127.1.8.65:46361/ using document root <none> and password file <none>
I20260806 22:27:39.236250 12270 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:39.236439 12270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:39.242655 12270 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:27:39.246409 12435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:39.247958 12270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 22:27:39.248194 12270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "4c99f6382fd44e2e92cf342c70c22db9"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "d46c62a5fb23a5dbd191a1833a36d6f3"
server_key_iv: "391d0a2d5a379e41c4ee6ffce1e6b661"
server_key_version: "encryptionkey@0"
I20260806 22:27:39.249370 12270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:39.275072 12270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:39.279862 12270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:39.280359 12270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:39.282120 12270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:39.284303 12270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:39.284461 12270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:39.284597 12270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:39.284729 12270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:39.362500 12270 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:40025
I20260806 22:27:39.362607 12604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:40025 every 8 connection(s)
I20260806 22:27:39.364414 12270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:39.375039 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 12270
I20260806 22:27:39.375532 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 11363
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.66@KRBTEST.COM
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:39.423167 12625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.377944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.1.8.66:39213)
0806 22:27:39.381324 (+ 3380us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:39.381339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:39.390482 (+ 9143us) server_negotiation.cc:407] Connection header received
0806 22:27:39.390722 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:39.390734 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:39.391058 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:39.391257 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:39.395547 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.397952 (+ 2405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.400693 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.401910 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.402090 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.410951 (+ 8861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:39.410976 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:39.410996 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:39.411045 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:39.417619 (+ 6574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.419059 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.419070 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.419074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.419320 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.420018 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.420024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.420028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.420602 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:39.421104 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:39.421874 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 22:27:39.422289 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3159,"thread_start_us":2918,"threads_started":1}
I20260806 22:27:39.425725 11560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:39.433290 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.369201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:53147 (local address 127.1.8.126:33507)
0806 22:27:39.370382 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:39.370389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:39.405512 (+ 35123us) server_negotiation.cc:407] Connection header received
0806 22:27:39.405727 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:39.405733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:39.405871 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:39.406069 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:39.408719 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.410260 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.414835 (+ 4575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.415838 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.415971 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.421449 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:39.421455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:39.421458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:39.421497 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:39.427976 (+ 6479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.428965 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.428971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.428975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.429268 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.429974 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.429981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.429986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.430480 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:39.430725 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:39.432239 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0806 22:27:39.432598 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":643,"threads_started":1}
I20260806 22:27:39.433396 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.369388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:56397 (local address 127.1.8.124:33417)
0806 22:27:39.369650 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:39.369659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:39.396819 (+ 27160us) server_negotiation.cc:407] Connection header received
0806 22:27:39.397255 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:39.397266 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:39.397407 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:39.397555 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:39.409207 (+ 11652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.410473 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.412956 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.413810 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.413920 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.423957 (+ 10037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:39.423974 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:39.423979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:39.424044 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:39.429754 (+ 5710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.430688 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.430696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.430699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.430901 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.431413 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.431417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.431420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.431751 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:39.431977 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:39.432539 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 22:27:39.432825 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:27:39.435148 12622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.371848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.1.8.65:56397)
0806 22:27:39.396689 (+ 24841us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:39.396713 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:39.397011 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:39.398120 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:39.398129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:39.404783 (+ 6654us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:39.408986 (+ 4203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.409004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.410796 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.410802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.412693 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.412711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.414377 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.414384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.414655 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.416766 (+ 2111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:39.416792 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:39.423662 (+ 6870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:39.429915 (+ 6253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.429921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.429924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.430510 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.431033 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.431036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.431038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.431214 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.431882 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:39.431887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:39.432175 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 22:27:39.433700 (+ 1525us) client_negotiation.cc:240] Negotiation successful
0806 22:27:39.434032 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24582,"mutex_wait_us":39,"spinlock_wait_cycles":1280,"thread_start_us":24135,"threads_started":1}
I20260806 22:27:39.435175 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.371873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:53147)
0806 22:27:39.405389 (+ 33516us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:39.405413 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:39.405557 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:39.406689 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:39.406696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:39.407869 (+ 1173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:39.408547 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.408563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.412015 (+ 3452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.412021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.414578 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.414598 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.416318 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.416327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.416601 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.417581 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:39.417604 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:39.421170 (+ 3566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:39.428142 (+ 6972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.428149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.428152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.428813 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.429421 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.429427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.429431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.429838 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.430675 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:39.430687 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:39.431402 (+ 715us) client_negotiation.cc:769] Sending connection context
0806 22:27:39.432070 (+ 668us) client_negotiation.cc:240] Negotiation successful
0806 22:27:39.432998 (+ 928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32618,"mutex_wait_us":324,"spinlock_wait_cycles":34048,"thread_start_us":26393,"threads_started":1}
I20260806 22:27:39.437434 12608 heartbeater.cc:347] Connected to a master server at 127.1.8.124:33417
I20260806 22:27:39.437939 12608 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:39.439402 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.370791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:35047 (local address 127.1.8.125:43001)
0806 22:27:39.373455 (+ 2664us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:39.373464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:39.391686 (+ 18222us) server_negotiation.cc:407] Connection header received
0806 22:27:39.397374 (+ 5688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:39.397381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:39.397490 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:39.397574 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:39.406746 (+ 9172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.408383 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.411979 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.413167 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.413323 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.420609 (+ 7286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:39.420618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:39.420623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:39.420672 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:39.427108 (+ 6436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.430047 (+ 2939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.430057 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.430063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.430343 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:39.435352 (+ 5009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:39.435361 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:39.435369 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:39.435947 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:39.436164 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:39.438537 (+ 2373us) server_negotiation.cc:299] Negotiation successful
0806 22:27:39.438859 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2430,"thread_start_us":2202,"threads_started":1}
I20260806 22:27:39.439328 12610 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:39.439581 12610 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:39.439841 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:39.371848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.1.8.65:35047)
0806 22:27:39.389518 (+ 17670us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:39.389592 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:39.395538 (+ 5946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:39.398099 (+ 2561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:39.398113 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:39.399772 (+ 1659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:39.406398 (+ 6626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.406437 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.409184 (+ 2747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.409190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.411793 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:39.411810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:39.413879 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:39.413888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:39.414356 (+ 468us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:39.415244 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:39.415286 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:39.420274 (+ 4988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:39.427310 (+ 7036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.427322 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.427342 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.429284 (+ 1942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.430525 (+ 1241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:39.430533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:39.434704 (+ 4171us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:39.435049 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:39.437833 (+ 2784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:39.437841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:39.438131 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 22:27:39.438315 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 22:27:39.438833 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17079,"mutex_wait_us":33,"thread_start_us":15812,"threads_started":1}
I20260806 22:27:39.440346 12610 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:39.441321 12611 heartbeater.cc:347] Connected to a master server at 127.1.8.125:43001
I20260806 22:27:39.441470 12611 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:39.441601 12608 heartbeater.cc:511] Master 127.1.8.124:33417 requested a full tablet report, sending...
I20260806 22:27:39.442440 12611 heartbeater.cc:511] Master 127.1.8.125:43001 requested a full tablet report, sending...
I20260806 22:27:39.442973 11053 ts_manager.cc:194] Re-registered known tserver with Master: 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025)
I20260806 22:27:39.444710 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:53147
I20260806 22:27:39.444587 12016 ts_manager.cc:194] Registered new tserver with Master: 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025)
I20260806 22:27:39.446973 12316 ts_manager.cc:194] Registered new tserver with Master: 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025)
I20260806 22:27:39.536191 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 11363
I20260806 22:27:39.565150 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:39213
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:33507
--webserver_port=34835
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:39.877576 12395 catalog_manager.cc:1259] Loaded cluster ID: 69a518de17674a02819485d85bb3de2f
I20260806 22:27:39.877735 12395 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: loading cluster ID for follower catalog manager: success
I20260806 22:27:39.880201 12395 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: acquiring CA information for follower catalog manager: success
I20260806 22:27:39.881910 12395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:27:39.928505 12630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:39.928789 12630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:39.928962 12630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:39.937071 12630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:39.937199 12630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:39.937371 12630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:39.947688 12630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:39213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=34835
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:39.949076 12630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:39.950445 12630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:39.965227 12639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:39.965179 12643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:39.965704 12638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:39.965826 12630 server_base.cc:1034] running on GCE node
I20260806 22:27:39.966578 12630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:39.967485 12630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:39.968695 12630 hybrid_clock.cc:648] HybridClock initialized: now 1786055259968623 us; error 53 us; skew 500 ppm
I20260806 22:27:39.969081 12630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:39 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:39.972538 12630 init.cc:377] Logged in from keytab as oryx/127.1.8.66@KRBTEST.COM (short username oryx)
I20260806 22:27:39.973996 12630 webserver.cc:460] Webserver started at http://127.1.8.66:34835/ using document root <none> and password file <none>
I20260806 22:27:39.974650 12630 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:39.974768 12630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:39.980327 12630 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:27:39.983080 12653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:39.984560 12630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:39.984750 12630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "90de8ee9d5054e64941641985d35adea"
format_stamp: "Formatted at 2026-08-06 22:27:35 on dist-test-slave-7vjv"
server_key: "e9060eda7f7cc4acd6f33907be1b2df1"
server_key_iv: "5a72cc5c732817c1a6d9712932689604"
server_key_version: "encryptionkey@0"
I20260806 22:27:39.985767 12630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:40.001612 12630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:40.006423 12630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:40.006755 12630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:40.007805 12630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:40.009295 12630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:40.009423 12630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:40.009560 12630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:40.009660 12630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:40.046192 12630 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39213
I20260806 22:27:40.046238 12823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39213 every 8 connection(s)
I20260806 22:27:40.047688 12630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:40.053812 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 12630
I20260806 22:27:40.054154 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 11578
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.67@KRBTEST.COM
I20260806 22:27:40.094175 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.050979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:38273 (local address 127.1.8.126:33507)
0806 22:27:40.069096 (+ 18117us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.069100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.072903 (+ 3803us) server_negotiation.cc:407] Connection header received
0806 22:27:40.073140 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.073146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.073242 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.073327 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.075266 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.076493 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.078995 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.079678 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.079775 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.085624 (+ 5849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.085632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.085636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.085679 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.089532 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091085 (+ 1553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.091090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.091094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.091255 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091814 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.091822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.091826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.092253 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.092377 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.093530 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.093760 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18003,"thread_start_us":17900,"threads_started":1}
I20260806 22:27:40.094213 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.051333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:46429 (local address 127.1.8.124:33417)
0806 22:27:40.068699 (+ 17366us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.068709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.075093 (+ 6384us) server_negotiation.cc:407] Connection header received
0806 22:27:40.075224 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.075232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.075322 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.075437 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.076846 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.078134 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.080068 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.080723 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.080839 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.086393 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.086399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.086404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.086445 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.090527 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091403 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.091408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.091412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.091557 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.092077 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.092082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.092085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.092508 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.092688 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.093515 (+ 827us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.093781 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17189,"thread_start_us":17030,"threads_started":1}
I20260806 22:27:40.095335 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.050834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:40627 (local address 127.1.8.125:43001)
0806 22:27:40.066570 (+ 15736us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.066589 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.069915 (+ 3326us) server_negotiation.cc:407] Connection header received
0806 22:27:40.072313 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.072319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.072426 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.072556 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.075582 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.076718 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.079027 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.079731 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.079833 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.087265 (+ 7432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.087274 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.087279 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.087335 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.091351 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.092280 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.092287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.092291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.092450 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.093004 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.093009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.093013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.093447 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.093571 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.094192 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.094941 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15543,"thread_start_us":15741,"threads_started":1}
I20260806 22:27:40.096530 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.051717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:38273)
0806 22:27:40.072809 (+ 21092us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.072827 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.072998 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.073541 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.073545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.074180 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.075104 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.075132 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.076704 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.076709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.078834 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.078849 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.080128 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.080139 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.080459 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.081781 (+ 1322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.081817 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.085377 (+ 3560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.089699 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.089707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.089724 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.090933 (+ 1209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.091407 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.091414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.091676 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.092509 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.092517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.092951 (+ 434us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.094391 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.095060 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20953,"mutex_wait_us":247,"spinlock_wait_cycles":8064,"thread_start_us":20788,"threads_started":1}
I20260806 22:27:40.096585 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.051426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.1.8.66:46429)
0806 22:27:40.074953 (+ 23527us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.074978 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.075101 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.075665 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.075672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.076316 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.076644 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.076660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.078314 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.078319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.079878 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.079891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.081030 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.081038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.081256 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.081935 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.081966 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.086182 (+ 4216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.090726 (+ 4544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.090731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.090734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.091270 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.091700 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.091713 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.091927 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.092761 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.092765 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.092951 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.094391 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.094944 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23360,"mutex_wait_us":547,"spinlock_wait_cycles":34176,"thread_start_us":17298,"threads_started":1}
I20260806 22:27:40.097047 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.051426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.1.8.66:40627)
0806 22:27:40.069700 (+ 18274us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.069749 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.071955 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.073012 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.073026 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.073987 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.075421 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.075433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.077274 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.077281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.078850 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.078869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.080160 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.080167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.080469 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.082425 (+ 1956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.082450 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.087047 (+ 4597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.091537 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.091543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.091546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.092141 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.092615 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.092619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.092622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.092870 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.093668 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.093672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.093848 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.095826 (+ 1978us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.096167 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17966,"mutex_wait_us":25,"spinlock_wait_cycles":16128,"thread_start_us":23072,"threads_started":1}
I20260806 22:27:40.096992 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.055817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.1.8.67:43835)
0806 22:27:40.073236 (+ 17419us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.073247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.074051 (+ 804us) server_negotiation.cc:407] Connection header received
0806 22:27:40.074148 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.074160 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.074473 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.074673 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.077211 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.078740 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.080693 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.081643 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.081766 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.088496 (+ 6730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.088510 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.088528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.088575 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.092605 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.093476 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.093488 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.093493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.093656 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.094125 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.094129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.094132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.094737 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.095050 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.096270 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.096534 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17235,"thread_start_us":17067,"threads_started":1}
I20260806 22:27:40.098224 12830 heartbeater.cc:347] Connected to a master server at 127.1.8.125:43001
I20260806 22:27:40.098304 12827 heartbeater.cc:347] Connected to a master server at 127.1.8.124:33417
I20260806 22:27:40.098604 12830 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.098605 12827 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.098964 11782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:40.098270 12828 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:40.099339 12828 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.099655 12827 heartbeater.cc:511] Master 127.1.8.124:33417 requested a full tablet report, sending...
I20260806 22:27:40.099844 12828 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:40.100241 12830 heartbeater.cc:511] Master 127.1.8.125:43001 requested a full tablet report, sending...
I20260806 22:27:40.101547 12316 ts_manager.cc:194] Registered new tserver with Master: 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:40.101979 12016 ts_manager.cc:194] Registered new tserver with Master: 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:40.103556 11053 ts_manager.cc:194] Re-registered known tserver with Master: 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:40.104542 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:38273
I20260806 22:27:40.183413 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 11578
I20260806 22:27:40.208372 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:43835
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:33507
--webserver_port=33153
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:40.450418 12610 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
W20260806 22:27:40.561041 12849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:40.561343 12849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:40.561517 12849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:40.569480 12849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:40.569630 12849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:40.569799 12849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:40.579888 12849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:43835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=33153
--tserver_master_addrs=127.1.8.126:33507,127.1.8.125:43001,127.1.8.124:33417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:40.581312 12849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:40.583047 12849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:40.597303 12854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:40.597515 12857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:40.597981 12860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:40.598817 12849 server_base.cc:1034] running on GCE node
I20260806 22:27:40.599654 12849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:40.600649 12849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:40.601857 12849 hybrid_clock.cc:648] HybridClock initialized: now 1786055260601820 us; error 25 us; skew 500 ppm
I20260806 22:27:40.602147 12849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055260, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:40.605497 12849 init.cc:377] Logged in from keytab as oryx/127.1.8.67@KRBTEST.COM (short username oryx)
I20260806 22:27:40.606957 12849 webserver.cc:460] Webserver started at http://127.1.8.67:33153/ using document root <none> and password file <none>
I20260806 22:27:40.607699 12849 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:40.607852 12849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:40.613309 12849 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 22:27:40.616173 12871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:40.617504 12849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:27:40.617695 12849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "49b3ef451a98414e95aca376cddd79ab"
format_stamp: "Formatted at 2026-08-06 22:27:36 on dist-test-slave-7vjv"
server_key: "4ccba93dbc602733c6dfa527ae6dbfef"
server_key_iv: "47eb010d4ab313eb358cb73103230e9c"
server_key_version: "encryptionkey@0"
I20260806 22:27:40.618700 12849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:40.638417 12849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:40.642953 12849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:40.643251 12849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:40.644361 12849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:40.645824 12849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:40.645944 12849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:40.646098 12849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:40.646188 12849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:40.680540 12849 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:43835
I20260806 22:27:40.680603 13052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:43835 every 8 connection(s)
I20260806 22:27:40.681993 12849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:40.686431 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 12849
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055260, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055260, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.124@KRBTEST.COM
Aug 06 22:27:40 dist-test-slave-7vjv krb5kdc[10977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055260, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.125@KRBTEST.COM
I20260806 22:27:40.736658 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.685553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.1.8.67:44717)
0806 22:27:40.712720 (+ 27167us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.712740 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.712844 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.716375 (+ 3531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.716383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.717151 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.717641 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.717658 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.719806 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.719811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.721523 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.721538 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.722740 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.722746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.722927 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.723589 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.723607 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.726826 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.731365 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.731376 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.731409 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.732171 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.732679 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.732685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.732690 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.733000 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.733844 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.733855 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.734309 (+ 454us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.734649 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.735179 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27011,"mutex_wait_us":84,"thread_start_us":27178,"threads_started":1}
I20260806 22:27:40.737164 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.685553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.1.8.67:34081)
0806 22:27:40.710003 (+ 24450us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.710031 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.711887 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.713207 (+ 1320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.713221 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.713918 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.714710 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.714731 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.718744 (+ 4013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.718749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.721136 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.721153 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.722443 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.722449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.722626 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.724789 (+ 2163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.724811 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.728965 (+ 4154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.733505 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.733510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.733514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.734063 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.734528 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.734533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.734536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.734672 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.735358 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.735363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.735508 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.735632 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.735910 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24272,"mutex_wait_us":60,"thread_start_us":24382,"threads_started":1}
I20260806 22:27:40.737792 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.684999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:34081 (local address 127.1.8.125:43001)
0806 22:27:40.711261 (+ 26262us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.711268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.711289 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:27:40.712452 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.712461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.712575 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.712700 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.714890 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.716931 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.721327 (+ 4396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.722001 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.722152 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.729225 (+ 7073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.729233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.729239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.729313 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.733351 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.734206 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.734211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.734213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.734385 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.734804 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.734808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.734810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.735142 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.735301 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.736755 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.736999 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26126,"thread_start_us":26278,"threads_started":1}
I20260806 22:27:40.738965 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.685168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:44717 (local address 127.1.8.124:33417)
0806 22:27:40.715803 (+ 30635us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.715812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.715840 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:27:40.715915 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.715923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.716035 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.717858 (+ 1823us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.718128 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.719556 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.721839 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.722537 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.722691 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.727107 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.727115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.727121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.727174 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.731176 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.732351 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.732359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.732364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.732536 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.733149 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.733154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.733157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.733616 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.733763 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.738365 (+ 4602us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.738618 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30462,"thread_start_us":31014,"threads_started":1}
I20260806 22:27:40.739009 13056 heartbeater.cc:347] Connected to a master server at 127.1.8.125:43001
I20260806 22:27:40.739336 13056 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.740368 13056 heartbeater.cc:511] Master 127.1.8.125:43001 requested a full tablet report, sending...
I20260806 22:27:40.741389 13054 heartbeater.cc:347] Connected to a master server at 127.1.8.124:33417
I20260806 22:27:40.741526 13054 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.742000 13054 heartbeater.cc:511] Master 127.1.8.124:33417 requested a full tablet report, sending...
I20260806 22:27:40.743377 12316 ts_manager.cc:194] Registered new tserver with Master: 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
I20260806 22:27:40.743610 12016 ts_manager.cc:194] Registered new tserver with Master: 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
I20260806 22:27:40.744596 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.685554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:52893)
0806 22:27:40.708973 (+ 23419us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.709060 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.711887 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.715194 (+ 3307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.715198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.715569 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:40.715764 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.715773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.717148 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.717153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.718552 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.718565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.719574 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.719582 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.719924 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.720852 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:40.720881 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:40.725690 (+ 4809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:40.741435 (+ 15745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.741441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.741447 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.742000 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.742571 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:40.742575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:40.742580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:40.742750 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:40.743423 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:40.743428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:40.743610 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.743735 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.744146 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22965,"thread_start_us":26152,"threads_started":1}
I20260806 22:27:40.746212 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.685316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:52893 (local address 127.1.8.126:33507)
0806 22:27:40.710779 (+ 25463us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.710787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.710812 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:27:40.714959 (+ 4147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.714967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.715058 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.715173 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.715920 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.717012 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.718711 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.719342 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.719482 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.726245 (+ 6763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.726251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.726256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.726302 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.741257 (+ 14955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.742240 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.742246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.742252 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.742436 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.742909 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.742913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.742916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.743268 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.745655 (+ 2387us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.745768 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.745963 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25328,"thread_start_us":25513,"threads_started":1}
I20260806 22:27:40.747080 13055 heartbeater.cc:347] Connected to a master server at 127.1.8.126:33507
I20260806 22:27:40.747206 13055 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:40.747836 13055 heartbeater.cc:511] Master 127.1.8.126:33507 requested a full tablet report, sending...
I20260806 22:27:40.748957 11053 ts_manager.cc:194] Re-registered known tserver with Master: 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
I20260806 22:27:40.750357 11053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:52893
I20260806 22:27:40.770677 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.745384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41206 (local address 127.1.8.124:33417)
0806 22:27:40.745545 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.745552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.749595 (+ 4043us) server_negotiation.cc:407] Connection header received
0806 22:27:40.749643 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.749647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.749731 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.749791 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.752680 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.753964 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.755738 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.756688 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.756836 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.761669 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.761676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.761682 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.761727 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.766295 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.767322 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.767328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.767331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.767507 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.767954 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.767959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.767963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.768625 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.768951 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.769529 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.770234 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:27:40.771184 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.734976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.1.8.125:43001)
0806 22:27:40.744776 (+ 9800us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.744781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.750568 (+ 5787us) server_negotiation.cc:407] Connection header received
0806 22:27:40.750632 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.750636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.750746 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.750846 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.754371 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.755788 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.758303 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.759088 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.759196 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.762991 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.762996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.762999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.763064 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.767346 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.768360 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.768365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.768381 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.768580 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.769055 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.769059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.769062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.769539 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.769911 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.770190 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.770462 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9643,"thread_start_us":9521,"threads_started":1}
I20260806 22:27:40.772619 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.721921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.1.8.126:33507)
0806 22:27:40.746496 (+ 24575us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.746502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.751614 (+ 5112us) server_negotiation.cc:407] Connection header received
0806 22:27:40.751673 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.751678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.751766 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.751878 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:40.755855 (+ 3977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.756976 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.758981 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.759745 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.759844 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.763953 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:40.763959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:40.763965 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:40.764006 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:40.767964 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.769003 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.769010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.769013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.769306 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:40.770222 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:40.770226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:40.770230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:40.770654 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:40.770979 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.772058 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.772338 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24478,"thread_start_us":24392,"threads_started":1}
I20260806 22:27:40.776984 11053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:40.785475 11053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:40.828014 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.809211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.1.8.66:39213)
0806 22:27:40.814269 (+ 5058us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.814278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.817998 (+ 3720us) server_negotiation.cc:407] Connection header received
0806 22:27:40.818136 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.818144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.818403 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.819874 (+ 1471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:40.820188 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.821639 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.824736 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.826126 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.826268 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.826542 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.827526 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.827711 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4841,"thread_start_us":4711,"threads_started":1}
I20260806 22:27:40.828166 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.810096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.1.8.67:43835)
0806 22:27:40.813414 (+ 3318us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.813428 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.817039 (+ 3611us) server_negotiation.cc:407] Connection header received
0806 22:27:40.817144 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.817150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.817378 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.817593 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:40.818621 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.820853 (+ 2232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.824779 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.826669 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.826884 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.827096 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.827466 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.827726 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3037,"thread_start_us":2628,"threads_started":1}
I20260806 22:27:40.828385 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.808948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39213 (local address 127.0.0.1:36892)
0806 22:27:40.816713 (+ 7765us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.816739 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.816934 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.818623 (+ 1689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.818627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.818642 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:40.819579 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.819607 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.821805 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.821811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.824578 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.824591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.826373 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.826379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.826550 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.826733 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.826832 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.827079 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7523,"mutex_wait_us":22,"thread_start_us":645,"threads_started":1}
I20260806 22:27:40.829380 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.809787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43835 (local address 127.0.0.1:42422)
0806 22:27:40.816718 (+ 6931us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.816739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.816895 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.817581 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.817585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.817610 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:40.818243 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.818259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.821034 (+ 2775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.821040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.824350 (+ 3310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.824366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.827034 (+ 2668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.827041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.827239 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.827272 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.827376 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.827522 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6756,"mutex_wait_us":1,"spinlock_wait_cycles":10624,"thread_start_us":3930,"threads_started":1}
I20260806 22:27:40.829833 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.809956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:40025 (local address 127.0.0.1:55314)
0806 22:27:40.816718 (+ 6762us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.816739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.816895 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.818607 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.818612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.818637 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:40.820188 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.820203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.822124 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.822130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.825038 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.825054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.827633 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.827642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.828897 (+ 1255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.828921 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.829033 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.829175 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6643,"mutex_wait_us":49,"spinlock_wait_cycles":1664,"thread_start_us":772,"threads_started":1}
I20260806 22:27:40.832949 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.810597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.1.8.65:40025)
0806 22:27:40.817640 (+ 7043us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.817649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.817680 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:27:40.817772 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.817778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.817960 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.818137 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:40.820410 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.821939 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.825207 (+ 3268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.826930 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.827054 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.827316 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.829097 (+ 1781us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.829270 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6885,"thread_start_us":6772,"threads_started":1}
I20260806 22:27:40.832706 12722 tablet_service.cc:1467] Processing CreateTablet for tablet 977789851f16433db0bba2ed55c80968 (DEFAULT_TABLE table=test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:40.834414 12722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977789851f16433db0bba2ed55c80968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:40.836200 12511 tablet_service.cc:1467] Processing CreateTablet for tablet 977789851f16433db0bba2ed55c80968 (DEFAULT_TABLE table=test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:40.836292 12953 tablet_service.cc:1467] Processing CreateTablet for tablet 977789851f16433db0bba2ed55c80968 (DEFAULT_TABLE table=test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:40.837994 12511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977789851f16433db0bba2ed55c80968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:40.838239 12953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977789851f16433db0bba2ed55c80968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:40.852952 13117 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Bootstrap starting.
I20260806 22:27:40.861460 13117 tablet_bootstrap.cc:654] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:40.863294 13120 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Bootstrap starting.
I20260806 22:27:40.864419 13117 log.cc:824] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:40.867816 13122 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Bootstrap starting.
I20260806 22:27:40.870715 13120 tablet_bootstrap.cc:654] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:40.873255 13120 log.cc:824] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:40.875723 13122 tablet_bootstrap.cc:654] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:40.877791 13120 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: No bootstrap required, opened a new log
I20260806 22:27:40.878211 13120 ts_tablet_manager.cc:1397] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260806 22:27:40.878258 13122 log.cc:824] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:40.878499 13117 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: No bootstrap required, opened a new log
I20260806 22:27:40.879019 13117 ts_tablet_manager.cc:1397] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.005s
I20260806 22:27:40.881798 13122 tablet_bootstrap.cc:492] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: No bootstrap required, opened a new log
I20260806 22:27:40.882205 13122 ts_tablet_manager.cc:1397] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260806 22:27:40.895840 13120 raft_consensus.cc:348] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.895843 13117 raft_consensus.cc:348] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.896600 13120 raft_consensus.cc:374] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:40.896582 13117 raft_consensus.cc:374] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:40.896793 13117 raft_consensus.cc:729] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c99f6382fd44e2e92cf342c70c22db9, State: Initialized, Role: FOLLOWER
I20260806 22:27:40.896780 13120 raft_consensus.cc:729] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de8ee9d5054e64941641985d35adea, State: Initialized, Role: FOLLOWER
I20260806 22:27:40.897770 13117 consensus_queue.cc:260] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.897773 13120 consensus_queue.cc:260] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.900074 13122 raft_consensus.cc:348] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.900818 13122 raft_consensus.cc:374] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:40.901117 13122 raft_consensus.cc:729] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b3ef451a98414e95aca376cddd79ab, State: Initialized, Role: FOLLOWER
I20260806 22:27:40.902112 13122 consensus_queue.cc:260] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.907383 13120 ts_tablet_manager.cc:1428] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Time spent starting tablet: real 0.029s user 0.022s sys 0.004s
I20260806 22:27:40.909924 13117 ts_tablet_manager.cc:1428] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Time spent starting tablet: real 0.031s user 0.020s sys 0.004s
I20260806 22:27:40.916069 12828 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
I20260806 22:27:40.921365 13134 raft_consensus.cc:482] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:40.921995 13134 raft_consensus.cc:504] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.922787 13055 heartbeater.cc:503] Master 127.1.8.126:33507 was elected leader, sending a full tablet report...
I20260806 22:27:40.926998 13122 ts_tablet_manager.cc:1428] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Time spent starting tablet: real 0.045s user 0.012s sys 0.019s
I20260806 22:27:40.927493 13134 leader_election.cc:288] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213), 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
W20260806 22:27:40.934998 13057 tablet.cc:2369] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:40.939592 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.925446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:52929 (local address 127.1.8.66:39213)
0806 22:27:40.926090 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.926098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.928720 (+ 2622us) server_negotiation.cc:407] Connection header received
0806 22:27:40.929037 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.929043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.929129 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.929195 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:40.931102 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.932422 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.936596 (+ 4174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.938444 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.938576 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.938680 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.939114 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.939274 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386}
I20260806 22:27:40.939651 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.924895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39213 (local address 127.1.8.65:52929)
0806 22:27:40.928612 (+ 3717us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.928633 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.928854 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.929430 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.929435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.929461 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:40.930031 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.930049 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.933228 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.933237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.936390 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.936405 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.938697 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.938706 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.938940 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.938965 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.939046 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.939146 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3539,"mutex_wait_us":21,"thread_start_us":3329,"threads_started":1}
I20260806 22:27:40.940788 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977789851f16433db0bba2ed55c80968" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de8ee9d5054e64941641985d35adea" is_pre_election: true
I20260806 22:27:40.941504 12751 raft_consensus.cc:2454] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 0.
I20260806 22:27:40.941879 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.929043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:52479 (local address 127.1.8.67:43835)
0806 22:27:40.929928 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:40.929937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:40.933295 (+ 3358us) server_negotiation.cc:407] Connection header received
0806 22:27:40.933629 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:40.933634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:40.933727 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:40.933797 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:40.935074 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.936370 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.939467 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.940762 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.940915 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.941062 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:40.941446 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 22:27:40.941576 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653}
I20260806 22:27:40.942229 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:40.928667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43835 (local address 127.1.8.65:52479)
0806 22:27:40.933182 (+ 4515us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:40.933201 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:40.933419 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:40.934033 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:40.934042 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:40.934071 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:40.934834 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.934868 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.936544 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.936549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.939272 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:40.939287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:40.941049 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:40.941055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:40.941280 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:40.941304 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:27:40.941417 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:27:40.941558 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4336,"mutex_wait_us":41,"thread_start_us":4079,"threads_started":1}
I20260806 22:27:40.942621 12438 leader_election.cc:302] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c99f6382fd44e2e92cf342c70c22db9, 90de8ee9d5054e64941641985d35adea; no voters:
I20260806 22:27:40.943190 13134 raft_consensus.cc:2781] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:40.943441 13134 raft_consensus.cc:482] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:40.943604 13134 raft_consensus.cc:3037] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:40.943733 12980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977789851f16433db0bba2ed55c80968" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b3ef451a98414e95aca376cddd79ab" is_pre_election: true
I20260806 22:27:40.944448 12980 raft_consensus.cc:2454] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 0.
I20260806 22:27:40.947038 13134 raft_consensus.cc:504] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.947829 13134 leader_election.cc:288] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 election: Requested vote from peers 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213), 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835)
I20260806 22:27:40.948498 12980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977789851f16433db0bba2ed55c80968" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b3ef451a98414e95aca376cddd79ab"
I20260806 22:27:40.948493 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977789851f16433db0bba2ed55c80968" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de8ee9d5054e64941641985d35adea"
I20260806 22:27:40.948757 12980 raft_consensus.cc:3037] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:40.948771 12751 raft_consensus.cc:3037] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:40.951647 12980 raft_consensus.cc:2454] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 1.
I20260806 22:27:40.951929 12751 raft_consensus.cc:2454] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 1.
I20260806 22:27:40.952281 12437 leader_election.cc:302] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b3ef451a98414e95aca376cddd79ab, 4c99f6382fd44e2e92cf342c70c22db9; no voters:
I20260806 22:27:40.952669 13134 raft_consensus.cc:2781] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:40.952978 13134 raft_consensus.cc:686] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 LEADER]: Becoming Leader. State: Replica: 4c99f6382fd44e2e92cf342c70c22db9, State: Running, Role: LEADER
I20260806 22:27:40.953715 13134 consensus_queue.cc:237] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } }
I20260806 22:27:40.960508 11051 catalog_manager.cc:5495] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "4c99f6382fd44e2e92cf342c70c22db9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } health_report { overall_health: HEALTHY } } }
I20260806 22:27:41.015493 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:41.009220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.1.8.65:40025)
0806 22:27:41.009420 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:41.009425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:41.009444 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:41.009605 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:41.009608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:41.009688 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:41.009815 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:41.010301 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.011332 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.013110 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.013624 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.013723 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:41.014285 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:41.014829 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:41.014985 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:41.015023 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:27:41.015084 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:27:41.025591 12751 raft_consensus.cc:1261] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Refusing update from remote peer 4c99f6382fd44e2e92cf342c70c22db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:41.025709 12980 raft_consensus.cc:1261] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Refusing update from remote peer 4c99f6382fd44e2e92cf342c70c22db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:41.026937 13140 consensus_queue.cc:1035] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:41.027585 13134 consensus_queue.cc:1035] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:41.047914 13148 mvcc.cc:205] Tried to move back new op lower bound from 7315682349148524544 to 7315682348871528448. Current Snapshot: MvccSnapshot[applied={T|T < 7315682349148524544}]
I20260806 22:27:41.053228 13150 mvcc.cc:205] Tried to move back new op lower bound from 7315682349148524544 to 7315682348871528448. Current Snapshot: MvccSnapshot[applied={T|T < 7315682349148524544}]
I20260806 22:27:41.084372 11051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:41.084786 11051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:41.093343 11051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7f980e5a1294880bcc4bb32f47bbe79: Sending DeleteTablet for 3 replicas of tablet 977789851f16433db0bba2ed55c80968
I20260806 22:27:41.094885 12953 tablet_service.cc:1514] Processing DeleteTablet for tablet 977789851f16433db0bba2ed55c80968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:41 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42422
I20260806 22:27:41.095369 12722 tablet_service.cc:1514] Processing DeleteTablet for tablet 977789851f16433db0bba2ed55c80968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:41 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:36892
I20260806 22:27:41.098901 12511 tablet_service.cc:1514] Processing DeleteTablet for tablet 977789851f16433db0bba2ed55c80968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:41 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55314
I20260806 22:27:41.102530 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 10999
I20260806 22:27:41.104166 11097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:41.108594 13164 tablet_replica.cc:330] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: stopping tablet replica
I20260806 22:27:41.109360 13164 raft_consensus.cc:2229] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:41.110322 13164 raft_consensus.cc:2258] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:41.112586 13163 tablet_replica.cc:330] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: stopping tablet replica
I20260806 22:27:41.113104 13163 raft_consensus.cc:2229] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:41.113473 13164 ts_tablet_manager.cc:1905] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:41.114078 13163 raft_consensus.cc:2258] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:41.118467 13166 tablet_replica.cc:330] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: stopping tablet replica
I20260806 22:27:41.120186 13163 ts_tablet_manager.cc:1905] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:41.123445 13164 ts_tablet_manager.cc:1918] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:41.123675 13164 log.cc:1196] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/977789851f16433db0bba2ed55c80968
I20260806 22:27:41.124760 13164 ts_tablet_manager.cc:1939] T 977789851f16433db0bba2ed55c80968 P 90de8ee9d5054e64941641985d35adea: Deleting consensus metadata
I20260806 22:27:41.126586 13166 raft_consensus.cc:2229] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:41.128154 13166 raft_consensus.cc:2258] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:41.130836 13166 ts_tablet_manager.cc:1905] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:41.133649 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:41.098096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.1.8.125:43001)
0806 22:27:41.098311 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:41.098320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:41.099885 (+ 1565us) server_negotiation.cc:407] Connection header received
0806 22:27:41.100149 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:41.100156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:41.100264 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:41.100406 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:41.112006 (+ 11600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.113545 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.117247 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.118133 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.118251 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:41.124127 (+ 5876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:41.124138 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:41.124143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:41.124191 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:41.129433 (+ 5242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.130549 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.130555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.130560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.130842 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.131407 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.131411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.131414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.131847 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:41.132314 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:41.132967 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 22:27:41.133209 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:27:41.134663 13163 ts_tablet_manager.cc:1918] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:41.141268 13166 ts_tablet_manager.cc:1918] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:41.141554 13166 log.cc:1196] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/977789851f16433db0bba2ed55c80968
I20260806 22:27:41.142606 13166 ts_tablet_manager.cc:1939] T 977789851f16433db0bba2ed55c80968 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting consensus metadata
I20260806 22:27:41.135006 13163 log.cc:1196] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/977789851f16433db0bba2ed55c80968
I20260806 22:27:41.149272 13163 ts_tablet_manager.cc:1939] T 977789851f16433db0bba2ed55c80968 P 49b3ef451a98414e95aca376cddd79ab: Deleting consensus metadata
I20260806 22:27:41.157160 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:41.100217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.1.8.124:33417)
0806 22:27:41.100399 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:41.100407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:41.135352 (+ 34945us) server_negotiation.cc:407] Connection header received
0806 22:27:41.135618 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:41.135624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:41.135738 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:41.135817 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:41.138877 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.140461 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.142748 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.143706 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.143813 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:41.148516 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:41.148522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:41.148527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:41.148568 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:41.153418 (+ 4850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.154465 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.154471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.154475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.154629 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.155060 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.155065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.155069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.155593 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:41.156163 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:41.156356 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:27:41.156579 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:27:41.257802 11029 catalog_manager.cc:4841] TS 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025): tablet 977789851f16433db0bba2ed55c80968 (table test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]) successfully deleted
I20260806 22:27:41.257750 11028 catalog_manager.cc:4841] TS 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835): tablet 977789851f16433db0bba2ed55c80968 (table test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]) successfully deleted
I20260806 22:27:41.258632 11029 catalog_manager.cc:4841] TS 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213): tablet 977789851f16433db0bba2ed55c80968 (table test-table [id=2e900ac8c96b4c6a9ed3dd418e3ac868]) successfully deleted
I20260806 22:27:41.258802 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 10999
I20260806 22:27:41.322103 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:41.298523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.1.8.124:33417)
0806 22:27:41.298789 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:41.298798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:41.303617 (+ 4819us) server_negotiation.cc:407] Connection header received
0806 22:27:41.304319 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:41.304330 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:41.304449 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:41.304559 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:41.308501 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.309692 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.311904 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.312455 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.312606 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:41.315898 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:41.315903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:41.315906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:41.315941 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:41.318972 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.319660 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.319667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.319671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.319813 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.320429 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.320436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.320440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.320783 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:41.321051 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:41.321422 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 22:27:41.321707 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:27:41.326925 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:41.297898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.1.8.125:43001)
0806 22:27:41.298194 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:41.298204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:41.308200 (+ 9996us) server_negotiation.cc:407] Connection header received
0806 22:27:41.308466 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:41.308471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:41.308562 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:41.308688 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:41.311520 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.312870 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.314865 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:41.316004 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:41.316212 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:41.320836 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:41.320845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:41.320852 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:41.320896 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:41.324227 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.324907 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.324912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.324916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.325047 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:41.325443 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:41.325448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:41.325453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:41.325734 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:41.325990 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:41.326352 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 22:27:41.326616 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:27:42.059441 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.058676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:36755)
0806 22:27:42.059186 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.059292 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
W20260806 22:27:42.060026 12828 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:33507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.068724 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.067987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:47867)
0806 22:27:42.068437 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.068555 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":259,"threads_started":1}
W20260806 22:27:42.069154 12610 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:33507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.073536 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.072685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:34229)
0806 22:27:42.073146 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.073304 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":247,"threads_started":1}
W20260806 22:27:42.073884 13055 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:33507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.594754 13186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7f980e5a1294880bcc4bb32f47bbe79)
I20260806 22:27:42.595088 13186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } }
I20260806 22:27:42.596635 13186 leader_election.cc:288] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507), 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125:43001)
I20260806 22:27:42.605394 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.596784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51278)
0806 22:27:42.604744 (+ 7960us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.605003 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7754,"thread_start_us":8243,"threads_started":1}
W20260806 22:27:42.606587 12291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.608682 13191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7f980e5a1294880bcc4bb32f47bbe79)
I20260806 22:27:42.609002 13191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } }
I20260806 22:27:42.609762 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.609365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51296)
0806 22:27:42.609530 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.609614 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:27:42.609938 12291 leader_election.cc:334] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507): Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.610656 13191 leader_election.cc:288] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507), 68d01b29806f487383336ffd1396148b (127.1.8.124:33417)
I20260806 22:27:42.611869 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.610741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51310)
0806 22:27:42.611261 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.611457 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":254,"threads_started":1}
W20260806 22:27:42.612149 11994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.614009 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.613677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51316)
0806 22:27:42.613813 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.613889 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:27:42.614295 11994 leader_election.cc:334] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507): Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.614894 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.596876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:43001 (local address 127.0.0.1:33224)
0806 22:27:42.606707 (+ 9831us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.606729 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:42.606903 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:42.607640 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:42.607649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:42.607696 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:42.608137 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.608152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.609781 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.609786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.612115 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.612132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.614180 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.614186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.614356 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.614407 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:27:42.614546 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:27:42.614678 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9697,"thread_start_us":9562,"threads_started":1}
I20260806 22:27:42.614899 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.596998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.1.8.125:43001)
0806 22:27:42.606279 (+ 9281us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.606287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.607055 (+ 768us) server_negotiation.cc:407] Connection header received
0806 22:27:42.607148 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.607156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.607269 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.607398 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:42.608366 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.609626 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.612266 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.613718 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.613840 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.614061 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.614584 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.614689 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":9156,"thread_start_us":8899,"threads_started":1}
I20260806 22:27:42.615612 12030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68d01b29806f487383336ffd1396148b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" is_pre_election: true
I20260806 22:27:42.615952 12030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d01b29806f487383336ffd1396148b in term 1.
I20260806 22:27:42.616545 12294 leader_election.cc:302] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56d35ca6da104f7eac8bbbcf3a37ef72, 68d01b29806f487383336ffd1396148b; no voters: a7f980e5a1294880bcc4bb32f47bbe79
I20260806 22:27:42.616967 13186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:27:42.617110 13186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7f980e5a1294880bcc4bb32f47bbe79)
I20260806 22:27:42.617233 13186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:27:42.619326 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.611028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.1.8.124:33417)
0806 22:27:42.611572 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.611577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.612133 (+ 556us) server_negotiation.cc:407] Connection header received
0806 22:27:42.612333 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.612340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.612413 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.612523 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:42.613375 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.614486 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.617444 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.618323 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.618425 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.618577 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.618862 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.619054 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":381,"threads_started":1}
I20260806 22:27:42.619354 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.611297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:33417 (local address 127.0.0.1:41232)
0806 22:27:42.611997 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.612016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:42.612176 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:42.612541 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:42.612545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:42.612575 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:42.613176 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.613189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.615114 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.615119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.617253 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.617270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.618498 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.618504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.618690 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.618725 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:27:42.618818 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:27:42.618951 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":59,"thread_start_us":428,"threads_started":1}
I20260806 22:27:42.619949 12333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68d01b29806f487383336ffd1396148b" is_pre_election: true
I20260806 22:27:42.620352 13186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } }
I20260806 22:27:42.620766 12333 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56d35ca6da104f7eac8bbbcf3a37ef72 in current term 2: Already voted for candidate 68d01b29806f487383336ffd1396148b in this term.
I20260806 22:27:42.621116 13186 leader_election.cc:288] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 election: Requested vote from peers a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507), 56d35ca6da104f7eac8bbbcf3a37ef72 (127.1.8.125:43001)
I20260806 22:27:42.621411 11997 leader_election.cc:302] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56d35ca6da104f7eac8bbbcf3a37ef72; no voters: 68d01b29806f487383336ffd1396148b, a7f980e5a1294880bcc4bb32f47bbe79
I20260806 22:27:42.621537 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.621163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51332)
0806 22:27:42.621335 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.621422 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 22:27:42.621639 12030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68d01b29806f487383336ffd1396148b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72"
I20260806 22:27:42.621840 13191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:27:42.622627 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.622334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51348)
0806 22:27:42.622456 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.622516 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:27:42.622787 12291 leader_election.cc:334] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507): Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
I20260806 22:27:42.624560 13191 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:27:42.626597 12030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d01b29806f487383336ffd1396148b in term 2.
I20260806 22:27:42.627312 12294 leader_election.cc:302] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56d35ca6da104f7eac8bbbcf3a37ef72, 68d01b29806f487383336ffd1396148b; no voters: a7f980e5a1294880bcc4bb32f47bbe79
I20260806 22:27:42.627600 13186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:27:42.628057 13186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [term 2 LEADER]: Becoming Leader. State: Replica: 68d01b29806f487383336ffd1396148b, State: Running, Role: LEADER
I20260806 22:27:42.628481 13186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } }
I20260806 22:27:42.630159 13203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68d01b29806f487383336ffd1396148b. Latest consensus state: current_term: 2 leader_uuid: "68d01b29806f487383336ffd1396148b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:42.630590 13203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:42.631286 13205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:42.635264 13205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:42.636165 13205 catalog_manager.cc:1259] Loaded cluster ID: 69a518de17674a02819485d85bb3de2f
I20260806 22:27:42.636261 13205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:42.637390 13205 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:42.638226 13205 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Loaded TSK: 0
I20260806 22:27:42.639045 13205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:27:42.679127 12316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:42.686331 12316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:42.691087 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.690564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51366)
0806 22:27:42.690814 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.690908 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 22:27:42.691071 12030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [term 2 FOLLOWER]: Refusing update from remote peer 68d01b29806f487383336ffd1396148b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 22:27:42.692008 13203 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [LEADER]: Connected to new peer: Peer: permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 22:27:42.692353 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.691878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51368)
0806 22:27:42.692040 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.692177 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:27:42.692591 12291 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b -> Peer a7f980e5a1294880bcc4bb32f47bbe79 (127.1.8.126:33507): Couldn't send request to peer a7f980e5a1294880bcc4bb32f47bbe79. Status: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:27:42.695124 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68d01b29806f487383336ffd1396148b. Latest consensus state: current_term: 2 leader_uuid: "68d01b29806f487383336ffd1396148b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:42.695429 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:42.697671 13186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "68d01b29806f487383336ffd1396148b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:42.698035 13186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:42.698566 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "68d01b29806f487383336ffd1396148b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a7f980e5a1294880bcc4bb32f47bbe79" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33507 } } peers { permanent_uuid: "56d35ca6da104f7eac8bbbcf3a37ef72" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 43001 } attrs { promote: false } } peers { permanent_uuid: "68d01b29806f487383336ffd1396148b" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 33417 } attrs { promote: false } } }
I20260806 22:27:42.698863 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d35ca6da104f7eac8bbbcf3a37ef72 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:27:42.714310 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.707732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:40025 (local address 127.0.0.1:55324)
0806 22:27:42.707877 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.707896 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:42.707994 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:42.708766 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:42.708770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:42.708794 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:42.708987 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.709001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.710247 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.710253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.711888 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.711900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.713516 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.713528 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.713704 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.713738 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:27:42.713826 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:27:42.713954 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:27:42.715199 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.707868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.1.8.65:40025)
0806 22:27:42.708451 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.708457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.708476 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:42.708571 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.708575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.708634 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.708745 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:42.709360 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.710106 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.712055 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.713314 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.713465 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.713589 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.714790 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.714942 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":315,"threads_started":1}
I20260806 22:27:42.715924 12511 tablet_service.cc:1467] Processing CreateTablet for tablet e11d35f530cf43ee8294efd137cd77e3 (DEFAULT_TABLE table=test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:42.716425 12511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11d35f530cf43ee8294efd137cd77e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:42.718863 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.710172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39213 (local address 127.0.0.1:36900)
0806 22:27:42.713360 (+ 3188us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.713384 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:42.713493 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:42.713857 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:42.713862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:42.713883 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:42.714255 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.714263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.715210 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.715217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.717024 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.717036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.718137 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.718143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.718310 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.718331 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:27:42.718429 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:27:42.718544 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3072,"thread_start_us":2858,"threads_started":1}
I20260806 22:27:42.718904 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.710747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.1.8.66:39213)
0806 22:27:42.711255 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.711261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.713602 (+ 2341us) server_negotiation.cc:407] Connection header received
0806 22:27:42.713680 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.713683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.713737 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.713846 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:42.714419 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.715074 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.717185 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.717921 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.718069 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.718190 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.718471 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.718610 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260806 22:27:42.719813 12722 tablet_service.cc:1467] Processing CreateTablet for tablet e11d35f530cf43ee8294efd137cd77e3 (DEFAULT_TABLE table=test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:42.720314 12722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11d35f530cf43ee8294efd137cd77e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:42.721663 13220 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Bootstrap starting.
I20260806 22:27:42.722690 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.708824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43835 (local address 127.0.0.1:42434)
0806 22:27:42.708969 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.708986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:42.709115 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:42.709983 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:42.709988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:42.710012 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:42.710353 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.710364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.712276 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.712284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.714575 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:42.714590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.722192 (+ 7602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.722196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:42.722308 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.722360 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:27:42.722424 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:27:42.722520 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:27:42.723533 13220 tablet_bootstrap.cc:654] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:42.726186 13220 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: No bootstrap required, opened a new log
I20260806 22:27:42.726336 13220 ts_tablet_manager.cc:1397] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 22:27:42.727057 13227 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Bootstrap starting.
I20260806 22:27:42.727355 13220 raft_consensus.cc:348] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.727566 13220 raft_consensus.cc:374] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:42.727659 13220 raft_consensus.cc:729] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c99f6382fd44e2e92cf342c70c22db9, State: Initialized, Role: FOLLOWER
I20260806 22:27:42.727918 13220 consensus_queue.cc:260] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.728857 13227 tablet_bootstrap.cc:654] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:42.732000 13227 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: No bootstrap required, opened a new log
I20260806 22:27:42.732149 13227 ts_tablet_manager.cc:1397] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:42.732852 13227 raft_consensus.cc:348] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.733062 13227 raft_consensus.cc:374] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:42.733147 13227 raft_consensus.cc:729] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de8ee9d5054e64941641985d35adea, State: Initialized, Role: FOLLOWER
I20260806 22:27:42.733402 13227 consensus_queue.cc:260] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.734906 13227 ts_tablet_manager.cc:1428] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:42.734944 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.709169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.1.8.67:43835)
0806 22:27:42.709649 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.709655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.709673 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:27:42.709744 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.709749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.709831 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.709949 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:42.710674 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.712049 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.714789 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.721937 (+ 7148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.734264 (+ 12327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.734402 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.734465 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.734632 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":281,"threads_started":1}
I20260806 22:27:42.735682 12953 tablet_service.cc:1467] Processing CreateTablet for tablet e11d35f530cf43ee8294efd137cd77e3 (DEFAULT_TABLE table=test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:42.736220 12953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11d35f530cf43ee8294efd137cd77e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:42.737622 12827 heartbeater.cc:503] Master 127.1.8.124:33417 was elected leader, sending a full tablet report...
I20260806 22:27:42.742333 13233 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Bootstrap starting.
I20260806 22:27:42.744045 13233 tablet_bootstrap.cc:654] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:42.744846 13220 ts_tablet_manager.cc:1428] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Time spent starting tablet: real 0.018s user 0.003s sys 0.007s
I20260806 22:27:42.746340 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.734578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:49719)
0806 22:27:42.745895 (+ 11317us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.746043 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11158,"threads_started":1}
I20260806 22:27:42.747673 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.746052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:52423)
0806 22:27:42.747238 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.747397 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1057,"threads_started":1}
I20260806 22:27:42.749145 12608 heartbeater.cc:503] Master 127.1.8.124:33417 was elected leader, sending a full tablet report...
I20260806 22:27:42.750569 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.750228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:58869)
0806 22:27:42.750365 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.750442 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:27:42.750844 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.748334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:48459)
0806 22:27:42.750507 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.750603 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2068}
I20260806 22:27:42.752701 13233 tablet_bootstrap.cc:492] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: No bootstrap required, opened a new log
I20260806 22:27:42.752835 13233 ts_tablet_manager.cc:1397] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260806 22:27:42.753608 13233 raft_consensus.cc:348] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.753808 13233 raft_consensus.cc:374] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:42.753928 13233 raft_consensus.cc:729] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b3ef451a98414e95aca376cddd79ab, State: Initialized, Role: FOLLOWER
I20260806 22:27:42.754215 13233 consensus_queue.cc:260] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.754992 13233 ts_tablet_manager.cc:1428] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:27:42.756341 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.755280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:46513)
0806 22:27:42.755945 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.756065 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":329,"threads_started":1}
I20260806 22:27:42.757335 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.756944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:50833)
0806 22:27:42.757108 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.757193 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:27:42.757426 13054 heartbeater.cc:503] Master 127.1.8.124:33417 was elected leader, sending a full tablet report...
W20260806 22:27:42.821005 12831 tablet.cc:2369] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:27:42.876327 12616 tablet.cc:2369] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:42.889183 13228 raft_consensus.cc:482] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:42.889448 13228 raft_consensus.cc:504] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.890334 13228 leader_election.cc:288] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835), 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:42.891007 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11d35f530cf43ee8294efd137cd77e3" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de8ee9d5054e64941641985d35adea" is_pre_election: true
I20260806 22:27:42.890965 12978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11d35f530cf43ee8294efd137cd77e3" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b3ef451a98414e95aca376cddd79ab" is_pre_election: true
I20260806 22:27:42.891294 12751 raft_consensus.cc:2454] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 0.
I20260806 22:27:42.891295 12978 raft_consensus.cc:2454] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 0.
I20260806 22:27:42.891796 12438 leader_election.cc:302] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c99f6382fd44e2e92cf342c70c22db9, 90de8ee9d5054e64941641985d35adea; no voters:
I20260806 22:27:42.892217 13228 raft_consensus.cc:2781] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:42.892367 13228 raft_consensus.cc:482] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:42.892462 13228 raft_consensus.cc:3037] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:42.894856 13228 raft_consensus.cc:504] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.895321 13228 leader_election.cc:288] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 election: Requested vote from peers 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835), 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213)
I20260806 22:27:42.895828 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11d35f530cf43ee8294efd137cd77e3" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de8ee9d5054e64941641985d35adea"
I20260806 22:27:42.896023 12751 raft_consensus.cc:3037] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:42.896270 12978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11d35f530cf43ee8294efd137cd77e3" candidate_uuid: "4c99f6382fd44e2e92cf342c70c22db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b3ef451a98414e95aca376cddd79ab"
I20260806 22:27:42.896479 12978 raft_consensus.cc:3037] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:42.899153 12751 raft_consensus.cc:2454] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 1.
I20260806 22:27:42.899493 12978 raft_consensus.cc:2454] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c99f6382fd44e2e92cf342c70c22db9 in term 1.
I20260806 22:27:42.899643 12438 leader_election.cc:302] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c99f6382fd44e2e92cf342c70c22db9, 90de8ee9d5054e64941641985d35adea; no voters:
I20260806 22:27:42.899993 13228 raft_consensus.cc:2781] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:42.900470 13228 raft_consensus.cc:686] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 LEADER]: Becoming Leader. State: Replica: 4c99f6382fd44e2e92cf342c70c22db9, State: Running, Role: LEADER
I20260806 22:27:42.900796 13228 consensus_queue.cc:237] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } }
I20260806 22:27:42.901528 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.901101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:37467)
0806 22:27:42.901255 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.901331 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:27:42.903743 12316 catalog_manager.cc:5495] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "4c99f6382fd44e2e92cf342c70c22db9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c99f6382fd44e2e92cf342c70c22db9" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 40025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 } health_report { overall_health: UNKNOWN } } }
I20260806 22:27:42.956331 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.951332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.1.8.65:40025)
0806 22:27:42.951569 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.951576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.951597 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:27:42.951682 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.951688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.951766 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.951868 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:42.952287 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.953013 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.954497 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:42.954958 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:42.955131 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:42.955559 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:42.955666 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:42.955768 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:42.955920 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:27:42.956006 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:27:42.959661 12751 raft_consensus.cc:1261] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Refusing update from remote peer 4c99f6382fd44e2e92cf342c70c22db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:42.959689 12978 raft_consensus.cc:1261] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Refusing update from remote peer 4c99f6382fd44e2e92cf342c70c22db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:42.960440 13239 consensus_queue.cc:1035] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de8ee9d5054e64941641985d35adea" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:42.960809 13228 consensus_queue.cc:1035] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b3ef451a98414e95aca376cddd79ab" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:42.963533 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.962972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:46189)
0806 22:27:42.963183 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.963289 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 22:27:42.963840 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.963383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:54111)
0806 22:27:42.963575 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.963694 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:27:42.971164 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.970582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.66:47813)
0806 22:27:42.970899 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.970995 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260806 22:27:42.972499 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.971222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:41053)
0806 22:27:42.972254 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.972349 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":915}
I20260806 22:27:42.975126 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.974688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.65:49319)
0806 22:27:42.974844 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.974935 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:27:42.976114 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.975614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.1.8.67:35563)
0806 22:27:42.975821 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:42.975906 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 22:27:42.981679 12315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:42.984539 12315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:42.990553 12315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68d01b29806f487383336ffd1396148b: Sending DeleteTablet for 3 replicas of tablet e11d35f530cf43ee8294efd137cd77e3
I20260806 22:27:42.991861 12511 tablet_service.cc:1514] Processing DeleteTablet for tablet e11d35f530cf43ee8294efd137cd77e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:42 UTC) from {username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:55324
I20260806 22:27:42.992353 12953 tablet_service.cc:1514] Processing DeleteTablet for tablet e11d35f530cf43ee8294efd137cd77e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:42 UTC) from {username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:42434
I20260806 22:27:42.992659 13257 tablet_replica.cc:330] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: stopping tablet replica
I20260806 22:27:42.992908 13257 raft_consensus.cc:2229] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:42.993007 13258 tablet_replica.cc:330] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: stopping tablet replica
I20260806 22:27:42.993011 12722 tablet_service.cc:1514] Processing DeleteTablet for tablet e11d35f530cf43ee8294efd137cd77e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:42 UTC) from {username='oryx', principal='oryx/127.1.8.124@KRBTEST.COM'} at 127.0.0.1:36900
I20260806 22:27:42.993294 13258 raft_consensus.cc:2229] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:42.993402 13257 raft_consensus.cc:2258] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:42.993643 13258 raft_consensus.cc:2258] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:42.994966 13258 ts_tablet_manager.cc:1905] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:42.995065 13257 ts_tablet_manager.cc:1905] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:42.995513 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 12270
I20260806 22:27:42.998049 13259 tablet_replica.cc:330] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: stopping tablet replica
I20260806 22:27:42.998224 13259 raft_consensus.cc:2229] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:42.998471 13259 raft_consensus.cc:2258] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:43.001750 13259 ts_tablet_manager.cc:1905] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:43.003968 13258 ts_tablet_manager.cc:1918] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:43.004180 13258 log.cc:1196] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/e11d35f530cf43ee8294efd137cd77e3
I20260806 22:27:43.004171 13257 ts_tablet_manager.cc:1918] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:43.004369 13257 log.cc:1196] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/e11d35f530cf43ee8294efd137cd77e3
I20260806 22:27:43.004838 13257 ts_tablet_manager.cc:1939] T e11d35f530cf43ee8294efd137cd77e3 P 4c99f6382fd44e2e92cf342c70c22db9: Deleting consensus metadata
I20260806 22:27:43.005762 13258 ts_tablet_manager.cc:1939] T e11d35f530cf43ee8294efd137cd77e3 P 49b3ef451a98414e95aca376cddd79ab: Deleting consensus metadata
I20260806 22:27:43.006273 12293 catalog_manager.cc:4841] TS 4c99f6382fd44e2e92cf342c70c22db9 (127.1.8.65:40025): tablet e11d35f530cf43ee8294efd137cd77e3 (table test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]) successfully deleted
I20260806 22:27:43.006798 12291 catalog_manager.cc:4841] TS 49b3ef451a98414e95aca376cddd79ab (127.1.8.67:43835): tablet e11d35f530cf43ee8294efd137cd77e3 (table test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]) successfully deleted
I20260806 22:27:43.007942 13259 ts_tablet_manager.cc:1918] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:43.008056 13259 log.cc:1196] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/e11d35f530cf43ee8294efd137cd77e3
I20260806 22:27:43.008559 13259 ts_tablet_manager.cc:1939] T e11d35f530cf43ee8294efd137cd77e3 P 90de8ee9d5054e64941641985d35adea: Deleting consensus metadata
I20260806 22:27:43.009657 12293 catalog_manager.cc:4841] TS 90de8ee9d5054e64941641985d35adea (127.1.8.66:39213): tablet e11d35f530cf43ee8294efd137cd77e3 (table test-table [id=2b9102b6f74540889ef8c4bc5978ae0c]) successfully deleted
I20260806 22:27:43.014053 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:42.996274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.1.8.65:40025)
0806 22:27:42.996446 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:42.996455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:42.999078 (+ 2623us) server_negotiation.cc:407] Connection header received
0806 22:27:42.999263 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:42.999268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:42.999350 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:42.999497 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:43.001213 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.002359 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.004443 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.005106 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.005234 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:43.008591 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:43.008610 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:43.008632 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:43.008671 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:43.011619 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.012332 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.012339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.012341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.012468 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.012834 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.012837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.012839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.013161 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:43.013382 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:43.013652 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:27:43.013815 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:27:43.014727 12597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:43.147747 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 12270
I20260806 22:27:43.178229 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 12630
I20260806 22:27:43.191699 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:43.178981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.1.8.66:39213)
0806 22:27:43.179213 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:43.179218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:43.179237 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:43.179365 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:43.179369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:43.179434 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:43.179518 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:43.180877 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.181570 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.182739 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.183149 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.183242 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:43.185807 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:43.185821 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:43.185836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:43.185868 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:43.188907 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.189596 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.189603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.189605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.189719 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.190152 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.190159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.190162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.190530 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:43.190851 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:43.191123 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:27:43.191367 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:27:43.192471 12819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:43.266762 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:43.265951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51380)
0806 22:27:43.266441 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:43.266527 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260806 22:27:43.268039 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:43.267694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51384)
0806 22:27:43.267845 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:43.267922 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:27:43.288816 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 12630
I20260806 22:27:43.317351 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 12849
I20260806 22:27:43.330847 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:43.318202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.1.8.67:43835)
0806 22:27:43.318741 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:43.318746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:43.318762 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:27:43.318819 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:43.318823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:43.318882 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:43.318983 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:43.320439 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.321209 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.322407 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:43.322876 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:43.322970 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:43.325619 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:43.325653 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:43.325670 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:43.325701 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:43.328613 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.329198 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.329205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.329207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.329324 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:43.329640 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:43.329643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:43.329645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:43.329976 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:43.330263 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:43.330443 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:27:43.330611 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":317,"threads_started":1}
I20260806 22:27:43.331776 13046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:43.425493 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 12849
I20260806 22:27:43.451679 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 11796
I20260806 22:27:43.452970 12055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:43.572372 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 11796
I20260806 22:27:43.602422 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 11950
I20260806 22:27:43.603552 12368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:43.728615 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:43.728016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:33507 (local address 127.0.0.1:51390)
0806 22:27:43.728251 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:43.728397 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 22:27:43.728930 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 11950
2026-08-06T22:27:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10878 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:43 dist-test-slave-7vjv krb5kdc[13275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:43 dist-test-slave-7vjv krb5kdc[13275](info): set up 2 sockets
Aug 06 22:27:43 dist-test-slave-7vjv krb5kdc[13275](info): commencing operation
krb5kdc: starting...
W20260806 22:27:45.094817 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:45 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab.
Entry for principal oryx/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab.
I20260806 22:27:45.384330 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43921
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab
--principal=oryx/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:43921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:45.733379 13300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:45.733657 13300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:45.733747 13300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:45.740921 13300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:45.741024 13300 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:27:45.741070 13300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:45.741137 13300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:45.741179 13300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:45.741235 13300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:45.750962 13300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:43921
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.126.keytab
--principal=oryx/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:45.752267 13300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:45.753612 13300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:45.765759 13306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:45.765843 13312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:45.765818 13308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:45.766623 13300 server_base.cc:1034] running on GCE node
I20260806 22:27:45.767446 13300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:45.768911 13300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:45.770129 13300 hybrid_clock.cc:648] HybridClock initialized: now 1786055265770056 us; error 46 us; skew 500 ppm
I20260806 22:27:45.770488 13300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:45 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055265, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:45.773634 13300 init.cc:377] Logged in from keytab as oryx/127.1.8.126@KRBTEST.COM (short username oryx)
I20260806 22:27:45.775036 13300 webserver.cc:460] Webserver started at http://127.1.8.126:43749/ using document root <none> and password file <none>
I20260806 22:27:45.775730 13300 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:45.775846 13300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:45.776262 13300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:45.779326 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93"
format_stamp: "Formatted at 2026-08-06 22:27:45 on dist-test-slave-7vjv"
server_key: "da55f794e7bb8b7bf568e943cf3bfe07"
server_key_iv: "f143ed2be3fd39732a5f7d242b78c551"
server_key_version: "encryptionkey@0"
I20260806 22:27:45.780251 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93"
format_stamp: "Formatted at 2026-08-06 22:27:45 on dist-test-slave-7vjv"
server_key: "da55f794e7bb8b7bf568e943cf3bfe07"
server_key_iv: "f143ed2be3fd39732a5f7d242b78c551"
server_key_version: "encryptionkey@0"
I20260806 22:27:45.785336 13300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:27:45.788705 13321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:45.789952 13300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:45.790126 13300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93"
format_stamp: "Formatted at 2026-08-06 22:27:45 on dist-test-slave-7vjv"
server_key: "da55f794e7bb8b7bf568e943cf3bfe07"
server_key_iv: "f143ed2be3fd39732a5f7d242b78c551"
server_key_version: "encryptionkey@0"
I20260806 22:27:45.790347 13300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:45.802317 13300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:45.806118 13300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:45.806393 13300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:45.827932 13300 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:43921
I20260806 22:27:45.827955 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:43921 every 8 connection(s)
I20260806 22:27:45.829303 13300 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:45.832954 13420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:45.833560 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 13300
I20260806 22:27:45.833810 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:45.834704 1057 external_mini_cluster.cc:1428] Setting key f07fddbecd91a151df42c369e511d42d
I20260806 22:27:45.847121 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Bootstrap starting.
Aug 06 22:27:45 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:45.850785 13420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:45.852033 13420 log.cc:824] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:45.854840 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: No bootstrap required, opened a new log
I20260806 22:27:45.858309 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:45.836817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.1.8.126:43921)
0806 22:27:45.837624 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:45.837634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:45.837674 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:27:45.839498 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:45.839520 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:45.839910 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:45.840303 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:45.842092 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:45.843323 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:45.845173 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:45.845795 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:45.845919 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:45.850512 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:45.850520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:45.850533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:45.850569 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:45.853733 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:45.854414 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:45.854420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:45.854425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:45.854596 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:45.855024 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:45.855029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:45.855032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:45.855802 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:45.856133 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:45.856501 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:27:45.856951 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":391,"threads_started":1}
I20260806 22:27:45.863847 13420 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } }
I20260806 22:27:45.864212 13420 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:45.864311 13420 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9fb1cf75f44fd7b9dc97bcb92e9f93, State: Initialized, Role: FOLLOWER
I20260806 22:27:45.864805 13420 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } }
I20260806 22:27:45.865007 13420 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:45.865118 13420 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:45.865238 13420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:45.867503 13420 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } }
I20260806 22:27:45.867931 13420 leader_election.cc:302] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb9fb1cf75f44fd7b9dc97bcb92e9f93; no voters:
I20260806 22:27:45.868553 13420 leader_election.cc:288] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:45.868703 13425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:45.869472 13425 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [term 1 LEADER]: Becoming Leader. State: Replica: bb9fb1cf75f44fd7b9dc97bcb92e9f93, State: Running, Role: LEADER
I20260806 22:27:45.869947 13425 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } }
I20260806 22:27:45.870649 13420 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:45.872854 13429 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb9fb1cf75f44fd7b9dc97bcb92e9f93. Latest consensus state: current_term: 1 leader_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } } }
I20260806 22:27:45.873574 13429 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:45.873545 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9fb1cf75f44fd7b9dc97bcb92e9f93" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43921 } } }
I20260806 22:27:45.876428 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:45.877261 13438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:45.882689 13438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:45.891225 13438 catalog_manager.cc:1347] Generated new cluster ID: 35149f46dc3e43709f7374530665820b
I20260806 22:27:45.891337 13438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:45.898038 13438 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:45.899219 13438 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:45.909032 13438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Generated new TSK 0
I20260806 22:27:45.909763 13438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
I20260806 22:27:45.983403 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43921
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:46.337864 13454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:46.338351 13454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:46.338557 13454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:46.347766 13454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:46.347903 13454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:46.347990 13454 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:27:46.348074 13454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:46.358554 13454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:43921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:46.359989 13454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:46.361470 13454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:46.375592 13469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:46.375504 13465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:46.375598 13464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:46.376256 13454 server_base.cc:1034] running on GCE node
I20260806 22:27:46.377014 13454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:46.377954 13454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:46.379225 13454 hybrid_clock.cc:648] HybridClock initialized: now 1786055266379118 us; error 45 us; skew 500 ppm
I20260806 22:27:46.379575 13454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:46 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055266, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:46.383292 13454 init.cc:377] Logged in from keytab as oryx/127.1.8.65@KRBTEST.COM (short username oryx)
I20260806 22:27:46.384956 13454 webserver.cc:460] Webserver started at http://127.1.8.65:41267/ using document root <none> and password file <none>
I20260806 22:27:46.385717 13454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:46.385847 13454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:46.386207 13454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:46.389377 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "f5bb8a01de1643f5a9abec5a4c101474"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "79c5704a5458d833929832ca9fe9b268"
server_key_iv: "7bc398f77349cd6b35ac7711c6ff3497"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.390249 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "f5bb8a01de1643f5a9abec5a4c101474"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "79c5704a5458d833929832ca9fe9b268"
server_key_iv: "7bc398f77349cd6b35ac7711c6ff3497"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.396449 13454 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 22:27:46.401957 13478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:46.403754 13454 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:27:46.404022 13454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "f5bb8a01de1643f5a9abec5a4c101474"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "79c5704a5458d833929832ca9fe9b268"
server_key_iv: "7bc398f77349cd6b35ac7711c6ff3497"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.404394 13454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:46.420680 13454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:46.425357 13454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:46.425653 13454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:46.428514 13454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:46.428627 13454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:46.428785 13454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:46.428892 13454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:27:46 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055266, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:46.453066 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.429178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:37761 (local address 127.1.8.126:43921)
0806 22:27:46.429828 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:46.429835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:46.431001 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 22:27:46.434149 (+ 3148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:46.434156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:46.434256 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:46.434392 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:46.436787 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.438156 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.440661 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.441297 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.441446 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:46.445786 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:46.445795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:46.445799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:46.445846 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:46.449758 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:46.450566 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:46.450572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:46.450576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:46.450723 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:46.451178 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:46.451183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:46.451186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:46.451478 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:46.451648 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:46.452453 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 22:27:46.452686 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":383,"threads_started":1}
I20260806 22:27:46.454113 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.429814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.65:37761)
0806 22:27:46.430793 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:46.430858 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:46.433773 (+ 2915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:46.434688 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:46.434703 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:46.435634 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:46.436522 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:46.436560 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.438343 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.438351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:46.440446 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:46.440471 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.441597 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.441606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:46.441961 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:46.442918 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:46.442953 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:46.445449 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:46.449941 (+ 4492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:46.449948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:46.449961 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:46.450427 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:46.450844 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:46.450851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:46.450859 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:46.451050 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:46.451631 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:46.451639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:46.451993 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 22:27:46.452333 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 22:27:46.452910 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":380,"threads_started":1}
I20260806 22:27:46.478705 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.460442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:34417 (local address 127.1.8.126:43921)
0806 22:27:46.460635 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:46.460639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:46.461257 (+ 618us) server_negotiation.cc:407] Connection header received
0806 22:27:46.461464 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:46.461469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:46.461562 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:46.461678 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:46.463662 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.464929 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.467727 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.468233 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.468386 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:46.471167 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:46.471172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:46.471175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:46.471209 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:46.474946 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:46.476117 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:46.476124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:46.476129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:46.476264 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:46.476770 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:46.476777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:46.476781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:46.477203 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:46.477374 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:46.477990 (+ 616us) server_negotiation.cc:299] Negotiation successful
0806 22:27:46.478260 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:27:46.478940 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.460522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.65:34417)
0806 22:27:46.461152 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:46.461168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:46.461331 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:46.461745 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:46.461750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:46.462711 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:46.463431 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:46.463452 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.465145 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.465152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:46.467474 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:46.467499 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.468432 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.468438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:46.468665 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:46.469522 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:46.469554 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:46.470936 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:46.475100 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:46.475108 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:46.475115 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:46.475955 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:46.476406 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:46.476414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:46.476423 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:46.476623 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:46.477369 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:46.477376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:46.477668 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 22:27:46.477868 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 22:27:46.478477 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":360,"threads_started":1}
I20260806 22:27:46.481161 13454 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44351
I20260806 22:27:46.481243 13681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44351 every 8 connection(s)
I20260806 22:27:46.482470 13454 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:46.484877 13682 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43921
I20260806 22:27:46.485108 13682 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:46.485659 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 13454
I20260806 22:27:46.485783 13682 heartbeater.cc:511] Master 127.1.8.126:43921 requested a full tablet report, sending...
I20260806 22:27:46.485873 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:46.486608 1057 external_mini_cluster.cc:1428] Setting key 53ef5a607e72f219b8b218e0b5c39842
I20260806 22:27:46.488149 13348 ts_manager.cc:194] Registered new tserver with Master: f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351)
I20260806 22:27:46.490219 13348 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:37761
WARNING: no policy specified for oryx/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
I20260806 22:27:46.545238 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43921
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:46.901750 13693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:46.902097 13693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:46.902284 13693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:46.909698 13693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:46.909847 13693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:46.909996 13693 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:27:46.910138 13693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:46.919807 13693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:43921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:46.921144 13693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:46.922469 13693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:46.936750 13710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:46.936776 13707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:46.936753 13706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:46.937958 13693 server_base.cc:1034] running on GCE node
I20260806 22:27:46.938715 13693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:46.939734 13693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:46.940997 13693 hybrid_clock.cc:648] HybridClock initialized: now 1786055266940930 us; error 38 us; skew 500 ppm
I20260806 22:27:46.941339 13693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:46 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055266, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:46.944708 13693 init.cc:377] Logged in from keytab as oryx/127.1.8.66@KRBTEST.COM (short username oryx)
I20260806 22:27:46.946282 13693 webserver.cc:460] Webserver started at http://127.1.8.66:43917/ using document root <none> and password file <none>
I20260806 22:27:46.947029 13693 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:46.947165 13693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:46.947528 13693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:46.950698 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "bf9e6ca72f5541fd81e717ae560e6306"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "6d294138e267c1d5a8ad23510c332b3b"
server_key_iv: "e5402183a12881cae1eb6dea9dc46024"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.951540 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "bf9e6ca72f5541fd81e717ae560e6306"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "6d294138e267c1d5a8ad23510c332b3b"
server_key_iv: "e5402183a12881cae1eb6dea9dc46024"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.957149 13693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 22:27:46.960675 13721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:46.961979 13693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:27:46.962167 13693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "bf9e6ca72f5541fd81e717ae560e6306"
format_stamp: "Formatted at 2026-08-06 22:27:46 on dist-test-slave-7vjv"
server_key: "6d294138e267c1d5a8ad23510c332b3b"
server_key_iv: "e5402183a12881cae1eb6dea9dc46024"
server_key_version: "encryptionkey@0"
I20260806 22:27:46.962421 13693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:46.981511 13693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:46.986064 13693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:46.986354 13693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:46.989290 13693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:46.989429 13693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:46.989606 13693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:46.989689 13693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055266, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:47.016059 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.989648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:54791 (local address 127.1.8.126:43921)
0806 22:27:46.990119 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:46.990126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:46.991405 (+ 1279us) server_negotiation.cc:407] Connection header received
0806 22:27:46.993785 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:46.993794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:46.993923 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:46.994063 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:46.996318 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.997767 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.000403 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.000980 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.001126 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.006967 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:47.006976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:47.006982 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:47.007041 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:47.011505 (+ 4464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.012815 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.012825 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.012831 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.013050 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.013568 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.013576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.013582 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.014094 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:47.014288 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.015101 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.015462 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":234,"threads_started":1}
I20260806 22:27:47.017014 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:46.990277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.66:54791)
0806 22:27:46.991210 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:46.991266 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:46.993489 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:46.994375 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:46.994386 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:46.995160 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:46.996075 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:46.996104 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:46.997968 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:46.997979 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.000228 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.000241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.001267 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.001275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.001488 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.002284 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:47.002319 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:47.006592 (+ 4273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:47.011701 (+ 5109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.011712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.011740 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.012619 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.013194 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.013197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.013200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.013430 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.014248 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:47.014254 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:47.014664 (+ 410us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.014932 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.015400 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":428,"threads_started":1}
I20260806 22:27:47.043226 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.026136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:39103 (local address 127.1.8.126:43921)
0806 22:27:47.026403 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.026412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.027708 (+ 1296us) server_negotiation.cc:407] Connection header received
0806 22:27:47.027862 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.027870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.027978 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.028144 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:47.029128 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.030605 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.032062 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.032833 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.033006 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.035752 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:47.035757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:47.035761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:47.035801 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:47.039543 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.040769 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.040777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.040785 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.040982 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.041439 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.041447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.041453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.041935 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:47.042121 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.042564 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.042817 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:27:47.043344 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.026758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.66:39103)
0806 22:27:47.027588 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.027611 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.027735 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.028194 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.028198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.028675 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:47.028984 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.028994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.030782 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.030786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.031912 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.031929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.033050 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.033055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.033214 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.033965 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:47.033994 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:47.035555 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:47.039806 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.039814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.039819 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.040602 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.041148 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.041152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.041155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.041294 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.042073 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:47.042081 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:47.042284 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.042443 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.042796 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":470,"threads_started":1}
I20260806 22:27:47.046998 13693 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39267
I20260806 22:27:47.047752 13911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39267 every 8 connection(s)
I20260806 22:27:47.048542 13693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:47.050563 13913 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43921
I20260806 22:27:47.050793 13913 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:47.051407 13913 heartbeater.cc:511] Master 127.1.8.126:43921 requested a full tablet report, sending...
I20260806 22:27:47.052721 13348 ts_manager.cc:194] Registered new tserver with Master: bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267)
I20260806 22:27:47.053534 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 13693
I20260806 22:27:47.053517 13348 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:54791
I20260806 22:27:47.053722 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:47.054212 1057 external_mini_cluster.cc:1428] Setting key 47036b12c84debff8287097b26190111
WARNING: no policy specified for oryx/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
I20260806 22:27:47.114997 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43921
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:47.464752 13924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:47.465019 13924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:47.465124 13924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:47.473227 13924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:47.473346 13924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:47.473439 13924 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:27:47.473518 13924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:47.483493 13924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.8.126:43921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:47.484797 13924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:47.486150 13924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:27:47.493284 13682 heartbeater.cc:503] Master 127.1.8.126:43921 was elected leader, sending a full tablet report...
W20260806 22:27:47.499756 13934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:47.499706 13935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:47.499795 13937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:47.500584 13924 server_base.cc:1034] running on GCE node
I20260806 22:27:47.501175 13924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:47.502127 13924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:47.503379 13924 hybrid_clock.cc:648] HybridClock initialized: now 1786055267503307 us; error 40 us; skew 500 ppm
I20260806 22:27:47.503679 13924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055267, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:47.506845 13924 init.cc:377] Logged in from keytab as oryx/127.1.8.67@KRBTEST.COM (short username oryx)
I20260806 22:27:47.508317 13924 webserver.cc:460] Webserver started at http://127.1.8.67:43169/ using document root <none> and password file <none>
I20260806 22:27:47.509006 13924 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:47.509140 13924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:47.509541 13924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:47.512600 13924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
format_stamp: "Formatted at 2026-08-06 22:27:47 on dist-test-slave-7vjv"
server_key: "913ad5c1c441d748f5d25db751fa9cd1"
server_key_iv: "1813816584032529c85e33c0a32a8a96"
server_key_version: "encryptionkey@0"
I20260806 22:27:47.513419 13924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
format_stamp: "Formatted at 2026-08-06 22:27:47 on dist-test-slave-7vjv"
server_key: "913ad5c1c441d748f5d25db751fa9cd1"
server_key_iv: "1813816584032529c85e33c0a32a8a96"
server_key_version: "encryptionkey@0"
I20260806 22:27:47.518611 13924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:27:47.522028 13951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:47.523309 13924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:27:47.523485 13924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
format_stamp: "Formatted at 2026-08-06 22:27:47 on dist-test-slave-7vjv"
server_key: "913ad5c1c441d748f5d25db751fa9cd1"
server_key_iv: "1813816584032529c85e33c0a32a8a96"
server_key_version: "encryptionkey@0"
I20260806 22:27:47.523720 13924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:47.538739 13924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:47.542611 13924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:47.542876 13924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:47.545862 13924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:47.546036 13924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 22:27:47.546195 13924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:47.546276 13924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055267, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:47.572882 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.546815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.67:54663)
0806 22:27:47.547884 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.547944 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.550726 (+ 2782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.551664 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.551680 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.552573 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:47.553398 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.553434 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.554783 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.554789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.556257 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.556271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.557472 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.557483 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.557833 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.558687 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:47.558733 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:47.562258 (+ 3525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:47.567357 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.567370 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.567392 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.568311 (+ 919us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.568883 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.568889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.568895 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.569195 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.569910 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:47.569918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:47.570436 (+ 518us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.570906 (+ 470us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.571457 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":547,"threads_started":1}
I20260806 22:27:47.572964 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.546240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:54663 (local address 127.1.8.126:43921)
0806 22:27:47.548869 (+ 2629us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.548873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.548892 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:47.551111 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.551117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.551207 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.551314 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:47.553584 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.554636 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.556425 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.557152 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.557330 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.562632 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:47.562642 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:47.562650 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:47.562703 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:47.567155 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.568509 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.568518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.568524 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.568713 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.569336 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.569343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.569346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.569769 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:47.569944 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.572310 (+ 2366us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.572540 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2493,"spinlock_wait_cycles":13696,"thread_start_us":2469,"threads_started":1}
I20260806 22:27:47.592929 14085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.578720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.1.8.67:53661)
0806 22:27:47.579475 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.579494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.579676 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.580026 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.580036 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.580526 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:47.580812 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.580820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.581820 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.581825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.582946 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.582959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.584107 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.584119 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.584376 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.585114 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:47.585144 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:47.586261 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:47.590056 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.590061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.590065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.590660 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.591162 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:47.591165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:47.591168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:47.591289 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:47.592047 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:47.592050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:47.592261 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.592376 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.592603 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":389,"threads_started":1}
I20260806 22:27:47.593400 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.578658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:53661 (local address 127.1.8.126:43921)
0806 22:27:47.578822 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.578826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.579590 (+ 764us) server_negotiation.cc:407] Connection header received
0806 22:27:47.579816 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.579829 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.579887 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.579991 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:47.580962 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.581673 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.583089 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.583640 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.583766 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.586547 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:47.586556 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:47.586561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:47.586601 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:47.589907 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.590816 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.590824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.590830 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.591028 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.591398 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.591406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.591420 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.591912 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:47.592133 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.592553 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.592866 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:27:47.603812 14152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:43587 every 8 connection(s)
I20260806 22:27:47.603844 13924 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:43587
I20260806 22:27:47.605201 13924 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:47.607281 14154 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43921
I20260806 22:27:47.607549 14154 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:47.608269 14154 heartbeater.cc:511] Master 127.1.8.126:43921 requested a full tablet report, sending...
I20260806 22:27:47.609750 13348 ts_manager.cc:194] Registered new tserver with Master: 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:47.610514 13348 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:54663
I20260806 22:27:47.614238 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 13924
I20260806 22:27:47.614392 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:47.614776 1057 external_mini_cluster.cc:1428] Setting key bb10ffebee6bfd62dff8779d7bd0b6fb
I20260806 22:27:47.617568 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:27:47.687551 13963 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:47.673139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.1.8.126:43921)
0806 22:27:47.673361 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.673368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.673561 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:27:47.673768 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.673772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.673888 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.673973 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:47.675449 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.676299 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.677483 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.677857 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.677947 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.687335 (+ 9388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52956: BlockingRecv error: recv got EOF from 127.0.0.1:52956 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 22:27:47 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:47.703518 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.690628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.1.8.126:43921)
0806 22:27:47.690801 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.690805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.691080 (+ 275us) server_negotiation.cc:407] Connection header received
0806 22:27:47.691206 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.691210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.691262 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.691341 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:47.692474 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.693159 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.694454 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.694896 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.695035 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.698624 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:47.698629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:47.698632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:47.698662 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:47.701027 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.701647 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.701659 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.701661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.701782 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:47.702163 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:47.702168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:47.702171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:47.702444 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:47.702737 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.702999 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.703222 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:27:47.707456 13348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:47.714607 13348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:47.739403 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.727787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.1.8.65:44351)
0806 22:27:47.729711 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.729722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.729756 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:27:47.729868 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.729879 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.730176 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.730409 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:47.732443 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.734068 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.736988 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.738248 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.738384 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.738613 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.738911 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.739087 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":1588,"threads_started":1}
I20260806 22:27:47.739908 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.726970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44351 (local address 127.0.0.1:42498)
0806 22:27:47.727762 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.727805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.728019 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.730449 (+ 2430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.730460 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.730502 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:47.732187 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.732210 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.734286 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.734294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.736826 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.736839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.738431 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.738437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.738632 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.738685 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.738819 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.739521 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":356,"threads_started":1}
I20260806 22:27:47.741384 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.727297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39267 (local address 127.0.0.1:55708)
0806 22:27:47.729354 (+ 2057us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.729376 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.729492 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.732776 (+ 3284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.732784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.732816 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:47.733225 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.733234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.735111 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.735120 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.737356 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.737372 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.739323 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.739329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.740671 (+ 1342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.740726 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.740816 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.740958 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1882,"mutex_wait_us":33,"thread_start_us":1723,"threads_started":1}
I20260806 22:27:47.741410 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.729176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49852 (local address 127.1.8.67:43587)
0806 22:27:47.732338 (+ 3162us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.732365 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.732933 (+ 568us) server_negotiation.cc:407] Connection header received
0806 22:27:47.733130 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.733138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.733323 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.733540 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:47.734344 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.735961 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.738141 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.739541 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.739692 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.739906 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.740894 (+ 988us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.741074 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2983,"thread_start_us":1644,"threads_started":1}
I20260806 22:27:47.742120 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.728367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.1.8.66:39267)
0806 22:27:47.731889 (+ 3522us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.731897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.731932 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:27:47.732027 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.732034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.732288 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.732550 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:47.733379 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.734954 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.737549 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.738857 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.738978 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.739171 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:47.741653 (+ 2482us) server_negotiation.cc:299] Negotiation successful
0806 22:27:47.741820 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3362,"thread_start_us":3216,"threads_started":1}
I20260806 22:27:47.743767 13577 tablet_service.cc:1467] Processing CreateTablet for tablet 94f412dfea09491ca57c83a9b1c19802 (DEFAULT_TABLE table=test-table [id=a8115b042ccf4d959e38be23746a6b70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:47.744699 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.728358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43587 (local address 127.0.0.1:49852)
0806 22:27:47.732803 (+ 4445us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.732820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.732956 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.733526 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.733532 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.733557 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:47.734120 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.734135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.736140 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.736146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.737981 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.737992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.740063 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.740071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.740678 (+ 607us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:47.740728 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:27:47.740837 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:27:47.740988 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4324,"mutex_wait_us":455,"thread_start_us":4192,"threads_started":1}
I20260806 22:27:47.746263 13577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f412dfea09491ca57c83a9b1c19802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:47.747471 13796 tablet_service.cc:1467] Processing CreateTablet for tablet 94f412dfea09491ca57c83a9b1c19802 (DEFAULT_TABLE table=test-table [id=a8115b042ccf4d959e38be23746a6b70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:47.747642 14037 tablet_service.cc:1467] Processing CreateTablet for tablet 94f412dfea09491ca57c83a9b1c19802 (DEFAULT_TABLE table=test-table [id=a8115b042ccf4d959e38be23746a6b70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:47.748772 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f412dfea09491ca57c83a9b1c19802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:47.748836 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f412dfea09491ca57c83a9b1c19802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:47.760155 14195 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Bootstrap starting.
I20260806 22:27:47.762110 14196 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Bootstrap starting.
I20260806 22:27:47.763301 14200 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Bootstrap starting.
I20260806 22:27:47.764575 14195 tablet_bootstrap.cc:654] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:47.765605 14196 tablet_bootstrap.cc:654] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:47.765949 14195 log.cc:824] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:47.766913 14200 tablet_bootstrap.cc:654] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:47.766953 14196 log.cc:824] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:47.768052 14200 log.cc:824] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:47.771101 14200 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: No bootstrap required, opened a new log
I20260806 22:27:47.771418 14195 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: No bootstrap required, opened a new log
I20260806 22:27:47.771451 14200 ts_tablet_manager.cc:1397] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 22:27:47.771654 14196 tablet_bootstrap.cc:492] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: No bootstrap required, opened a new log
I20260806 22:27:47.771693 14195 ts_tablet_manager.cc:1397] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260806 22:27:47.771929 14196 ts_tablet_manager.cc:1397] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 22:27:47.779893 14200 raft_consensus.cc:348] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.780475 14200 raft_consensus.cc:374] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:47.780586 14200 raft_consensus.cc:729] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e6ca72f5541fd81e717ae560e6306, State: Initialized, Role: FOLLOWER
I20260806 22:27:47.781268 14196 raft_consensus.cc:348] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.781278 14200 consensus_queue.cc:260] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.781637 14196 raft_consensus.cc:374] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:47.781734 14196 raft_consensus.cc:729] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Initialized, Role: FOLLOWER
I20260806 22:27:47.782328 14196 consensus_queue.cc:260] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.784345 14196 ts_tablet_manager.cc:1428] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260806 22:27:47.785378 13913 heartbeater.cc:503] Master 127.1.8.126:43921 was elected leader, sending a full tablet report...
I20260806 22:27:47.786026 14200 ts_tablet_manager.cc:1428] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260806 22:27:47.784891 14195 raft_consensus.cc:348] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.786486 14195 raft_consensus.cc:374] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:47.786605 14195 raft_consensus.cc:729] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Initialized, Role: FOLLOWER
I20260806 22:27:47.787178 14195 consensus_queue.cc:260] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.788576 14154 heartbeater.cc:503] Master 127.1.8.126:43921 was elected leader, sending a full tablet report...
I20260806 22:27:47.790421 14195 ts_tablet_manager.cc:1428] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent starting tablet: real 0.019s user 0.004s sys 0.011s
W20260806 22:27:47.800637 13914 tablet.cc:2369] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:27:47.857064 14155 tablet.cc:2369] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:27:47.985616 13684 tablet.cc:2369] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:27:47.992053 14206 raft_consensus.cc:482] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:47.992375 14206 raft_consensus.cc:504] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:47.994252 14206 leader_election.cc:288] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:48.001361 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.994833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60781 (local address 127.1.8.67:43587)
0806 22:27:47.995090 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.995099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.995790 (+ 691us) server_negotiation.cc:407] Connection header received
0806 22:27:47.995972 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.995977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.996039 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.996185 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:47.996878 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.997612 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.999501 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.000366 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.000470 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.000613 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.000959 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.001089 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:27:48.001395 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.994609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:48617 (local address 127.1.8.66:39267)
0806 22:27:47.994873 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:47.994884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:47.995735 (+ 851us) server_negotiation.cc:407] Connection header received
0806 22:27:47.995984 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:47.995988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:47.996046 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:47.996189 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:47.996929 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.997649 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.999546 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.000391 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.000556 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.000662 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.000992 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.001128 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:27:48.001520 14209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.994441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39267 (local address 127.1.8.65:48617)
0806 22:27:47.995620 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.995638 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.995800 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.996212 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.996216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.996241 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:47.996743 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.996752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.997815 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.997818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.999361 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.999374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.000669 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.000674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.000790 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.000832 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.000951 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.001076 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":47,"thread_start_us":747,"threads_started":1}
I20260806 22:27:48.001904 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:47.994791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43587 (local address 127.1.8.65:60781)
0806 22:27:47.995651 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:47.995666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:47.995821 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:47.996200 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:47.996205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:47.996228 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:47.996699 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.996713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:47.997745 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:47.997754 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:47.999349 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:47.999361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.000603 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.000609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.000778 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.000822 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.000897 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.001037 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":84,"thread_start_us":528,"threads_started":1}
I20260806 22:27:48.002822 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f412dfea09491ca57c83a9b1c19802" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306" is_pre_election: true
I20260806 22:27:48.002861 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f412dfea09491ca57c83a9b1c19802" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" is_pre_election: true
I20260806 22:27:48.003535 14071 raft_consensus.cc:2454] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:48.003530 13829 raft_consensus.cc:2454] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:48.004307 13484 leader_election.cc:302] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, bf9e6ca72f5541fd81e717ae560e6306, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:48.004724 14206 raft_consensus.cc:2781] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:48.004884 14206 raft_consensus.cc:482] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:48.004997 14206 raft_consensus.cc:3037] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.007144 14206 raft_consensus.cc:504] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.007737 14206 leader_election.cc:288] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Requested vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:48.008241 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f412dfea09491ca57c83a9b1c19802" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306"
I20260806 22:27:48.008296 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f412dfea09491ca57c83a9b1c19802" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
I20260806 22:27:48.008477 13829 raft_consensus.cc:3037] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.008567 14071 raft_consensus.cc:3037] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.010880 13829 raft_consensus.cc:2454] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:48.011057 14071 raft_consensus.cc:2454] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:48.011340 13483 leader_election.cc:302] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf9e6ca72f5541fd81e717ae560e6306, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:48.011667 14206 raft_consensus.cc:2781] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:48.012274 14206 raft_consensus.cc:686] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Becoming Leader. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Running, Role: LEADER
I20260806 22:27:48.012789 14206 consensus_queue.cc:237] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.017282 13348 catalog_manager.cc:5495] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "f5bb8a01de1643f5a9abec5a4c101474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } health_report { overall_health: UNKNOWN } } }
I20260806 22:27:48.034742 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.029656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.1.8.65:44351)
0806 22:27:48.029867 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.029872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.029890 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:27:48.030075 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.030085 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.030163 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.030284 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:48.030597 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.031310 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.032638 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.033051 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.033159 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.033540 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.034117 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.034224 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.034319 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.034387 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:27:48.043047 14071 raft_consensus.cc:1261] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.043047 13829 raft_consensus.cc:1261] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.044015 14206 consensus_queue.cc:1035] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.044376 14213 consensus_queue.cc:1035] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.059155 14220 mvcc.cc:205] Tried to move back new op lower bound from 7315682377894920192 to 7315682377784336384. Current Snapshot: MvccSnapshot[applied={T|T < 7315682377894920192}]
I20260806 22:27:48.060686 14221 mvcc.cc:205] Tried to move back new op lower bound from 7315682377894920192 to 7315682377784336384. Current Snapshot: MvccSnapshot[applied={T|T < 7315682377894920192}]
I20260806 22:27:48.079308 13348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:48.079588 13348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:48.083410 13348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Sending DeleteTablet for 3 replicas of tablet 94f412dfea09491ca57c83a9b1c19802
I20260806 22:27:48.084520 13577 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f412dfea09491ca57c83a9b1c19802 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42498
I20260806 22:27:48.084590 13796 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f412dfea09491ca57c83a9b1c19802 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55708
I20260806 22:27:48.085444 14037 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f412dfea09491ca57c83a9b1c19802 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49852
I20260806 22:27:48.085840 14238 tablet_replica.cc:330] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: stopping tablet replica
I20260806 22:27:48.086419 14238 raft_consensus.cc:2229] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:48.087332 14238 raft_consensus.cc:2258] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.087214 13348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:48.087945 13348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:48.088222 14240 tablet_replica.cc:330] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: stopping tablet replica
I20260806 22:27:48.088971 14240 raft_consensus.cc:2229] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:48.089588 14240 raft_consensus.cc:2258] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.090489 14241 tablet_replica.cc:330] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: stopping tablet replica
I20260806 22:27:48.090939 14241 raft_consensus.cc:2229] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:48.091454 14241 raft_consensus.cc:2258] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.092456 14238 ts_tablet_manager.cc:1905] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.093528 14241 ts_tablet_manager.cc:1905] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.097152 13796 tablet_service.cc:1467] Processing CreateTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (DEFAULT_TABLE table=test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.097663 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8bd6b60be34c4b9ddb3250630ab5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.100703 14037 tablet_service.cc:1467] Processing CreateTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (DEFAULT_TABLE table=test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.101114 14241 ts_tablet_manager.cc:1918] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:48.101256 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8bd6b60be34c4b9ddb3250630ab5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.101342 14241 log.cc:1196] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/94f412dfea09491ca57c83a9b1c19802
I20260806 22:27:48.101491 14240 ts_tablet_manager.cc:1905] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.101949 14241 ts_tablet_manager.cc:1939] T 94f412dfea09491ca57c83a9b1c19802 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting consensus metadata
I20260806 22:27:48.103974 13328 catalog_manager.cc:4841] TS 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587): tablet 94f412dfea09491ca57c83a9b1c19802 (table test-table [id=a8115b042ccf4d959e38be23746a6b70]) successfully deleted
I20260806 22:27:48.104592 13577 tablet_service.cc:1467] Processing CreateTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (DEFAULT_TABLE table=test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.105032 13577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8bd6b60be34c4b9ddb3250630ab5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.110467 14240 ts_tablet_manager.cc:1918] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:48.110669 14240 log.cc:1196] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/94f412dfea09491ca57c83a9b1c19802
I20260806 22:27:48.111253 14240 ts_tablet_manager.cc:1939] T 94f412dfea09491ca57c83a9b1c19802 P bf9e6ca72f5541fd81e717ae560e6306: Deleting consensus metadata
I20260806 22:27:48.111428 14196 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Bootstrap starting.
I20260806 22:27:48.113111 13327 catalog_manager.cc:4841] TS bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267): tablet 94f412dfea09491ca57c83a9b1c19802 (table test-table [id=a8115b042ccf4d959e38be23746a6b70]) successfully deleted
I20260806 22:27:48.113325 14196 tablet_bootstrap.cc:654] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.115720 14196 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: No bootstrap required, opened a new log
I20260806 22:27:48.115885 14196 ts_tablet_manager.cc:1397] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 22:27:48.116829 14196 raft_consensus.cc:348] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.117075 14196 raft_consensus.cc:374] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.117189 14196 raft_consensus.cc:729] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.117482 14196 consensus_queue.cc:260] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.117762 14195 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Bootstrap starting.
I20260806 22:27:48.118906 14196 ts_tablet_manager.cc:1428] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:27:48.119613 14195 tablet_bootstrap.cc:654] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.123761 14200 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Bootstrap starting.
I20260806 22:27:48.124231 14238 ts_tablet_manager.cc:1918] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:48.124528 14238 log.cc:1196] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/94f412dfea09491ca57c83a9b1c19802
I20260806 22:27:48.125129 14238 ts_tablet_manager.cc:1939] T 94f412dfea09491ca57c83a9b1c19802 P f5bb8a01de1643f5a9abec5a4c101474: Deleting consensus metadata
I20260806 22:27:48.125546 14200 tablet_bootstrap.cc:654] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.127260 13323 catalog_manager.cc:4841] TS f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351): tablet 94f412dfea09491ca57c83a9b1c19802 (table test-table [id=a8115b042ccf4d959e38be23746a6b70]) successfully deleted
I20260806 22:27:48.129055 14195 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: No bootstrap required, opened a new log
I20260806 22:27:48.129168 14200 tablet_bootstrap.cc:492] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: No bootstrap required, opened a new log
I20260806 22:27:48.129245 14195 ts_tablet_manager.cc:1397] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260806 22:27:48.129365 14200 ts_tablet_manager.cc:1397] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 22:27:48.130031 14200 raft_consensus.cc:348] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.130236 14200 raft_consensus.cc:374] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.130340 14200 raft_consensus.cc:729] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e6ca72f5541fd81e717ae560e6306, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.130192 14195 raft_consensus.cc:348] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.130456 14195 raft_consensus.cc:374] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.130577 14195 raft_consensus.cc:729] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.130884 14195 consensus_queue.cc:260] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.130671 14200 consensus_queue.cc:260] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.132226 14195 ts_tablet_manager.cc:1428] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:27:48.133380 14200 ts_tablet_manager.cc:1428] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:27:48.296202 14206 raft_consensus.cc:482] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:48.296429 14206 raft_consensus.cc:504] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.297233 14206 leader_election.cc:288] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:48.297832 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8bd6b60be34c4b9ddb3250630ab5c0" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" is_pre_election: true
I20260806 22:27:48.297844 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8bd6b60be34c4b9ddb3250630ab5c0" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306" is_pre_election: true
I20260806 22:27:48.298102 14071 raft_consensus.cc:2454] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:48.298167 13829 raft_consensus.cc:2454] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:48.298615 13484 leader_election.cc:302] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:48.298951 14206 raft_consensus.cc:2781] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:48.299108 14206 raft_consensus.cc:482] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:48.299222 14206 raft_consensus.cc:3037] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.301887 14206 raft_consensus.cc:504] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.302598 14206 leader_election.cc:288] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Requested vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:48.303113 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8bd6b60be34c4b9ddb3250630ab5c0" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306"
I20260806 22:27:48.303354 13829 raft_consensus.cc:3037] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.303265 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8bd6b60be34c4b9ddb3250630ab5c0" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
I20260806 22:27:48.303536 14071 raft_consensus.cc:3037] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.305680 14071 raft_consensus.cc:2454] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:48.306187 13484 leader_election.cc:302] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:48.306383 13829 raft_consensus.cc:2454] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:48.306491 14206 raft_consensus.cc:2781] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:48.306674 14206 raft_consensus.cc:686] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Becoming Leader. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Running, Role: LEADER
I20260806 22:27:48.306993 14206 consensus_queue.cc:237] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.310180 13348 catalog_manager.cc:5495] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "f5bb8a01de1643f5a9abec5a4c101474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:27:48 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055265, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.126@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:48.388052 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.370694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43921 (local address 127.0.0.1:52972)
0806 22:27:48.371779 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.371802 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.371973 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.372385 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.372391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.373426 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:48.373714 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.373729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.375344 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.375351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.377082 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.377101 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.377838 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.377845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.378031 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.378772 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:48.378798 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:48.381751 (+ 2953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:48.384892 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:48.384905 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:48.384913 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:48.385554 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:48.385962 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:48.385967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:48.385971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:48.386194 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:48.386752 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:48.386761 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:48.386948 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.387209 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.387545 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":759,"threads_started":1}
I20260806 22:27:48.388231 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.370729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.1.8.126:43921)
0806 22:27:48.371609 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.371615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.371896 (+ 281us) server_negotiation.cc:407] Connection header received
0806 22:27:48.372142 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.372146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.372207 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.372311 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:48.373904 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.375134 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.377245 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.377651 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.377752 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.381947 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:48.381952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:48.381955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:48.381995 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:48.384753 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:48.385682 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:48.385686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:48.385688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:48.385821 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:48.386354 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:48.386357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:48.386359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:48.386597 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:48.386702 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.387279 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.387754 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":196,"server-negotiator.queue_time_us":657,"thread_start_us":384,"threads_started":1}
I20260806 22:27:48.394409 13348 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:52972:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:27:48.395697 13348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:48.402894 13577 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcd3810d11f459bb41654abe28502c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=721cdf4173644c9e9d42c81ded445607]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:27:48.403079 13796 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcd3810d11f459bb41654abe28502c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=721cdf4173644c9e9d42c81ded445607]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:27:48.403564 13577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcd3810d11f459bb41654abe28502c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.403443 14037 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcd3810d11f459bb41654abe28502c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=721cdf4173644c9e9d42c81ded445607]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:27:48.403712 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcd3810d11f459bb41654abe28502c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.404000 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcd3810d11f459bb41654abe28502c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.411696 14195 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: Bootstrap starting.
I20260806 22:27:48.411734 14200 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: Bootstrap starting.
I20260806 22:27:48.413482 14200 tablet_bootstrap.cc:654] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.413482 14195 tablet_bootstrap.cc:654] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.414662 14196 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: Bootstrap starting.
I20260806 22:27:48.416054 14195 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: No bootstrap required, opened a new log
I20260806 22:27:48.416270 14195 ts_tablet_manager.cc:1397] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 22:27:48.416553 14196 tablet_bootstrap.cc:654] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.417078 14195 raft_consensus.cc:348] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.417346 14195 raft_consensus.cc:374] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.417454 14195 raft_consensus.cc:729] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.417765 14195 consensus_queue.cc:260] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.418283 14207 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.418627 14207 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.419117 14195 ts_tablet_manager.cc:1428] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:27:48.419140 14200 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: No bootstrap required, opened a new log
I20260806 22:27:48.419353 14200 ts_tablet_manager.cc:1397] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 22:27:48.420230 14200 raft_consensus.cc:348] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.420569 14200 raft_consensus.cc:374] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.420671 14200 raft_consensus.cc:729] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e6ca72f5541fd81e717ae560e6306, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.420956 14200 consensus_queue.cc:260] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.421422 14205 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.421758 14205 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.422742 14200 ts_tablet_manager.cc:1428] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:27:48.424424 14196 tablet_bootstrap.cc:492] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: No bootstrap required, opened a new log
I20260806 22:27:48.424647 14196 ts_tablet_manager.cc:1397] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 22:27:48.425256 14196 raft_consensus.cc:348] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.425444 14196 raft_consensus.cc:374] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.425498 14196 raft_consensus.cc:729] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.425755 14196 consensus_queue.cc:260] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.426499 14196 ts_tablet_manager.cc:1428] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 22:27:48.426340 14206 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.426707 14206 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.593801 14207 raft_consensus.cc:482] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:48.594053 14207 raft_consensus.cc:504] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.595657 14207 leader_election.cc:288] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351), bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267)
I20260806 22:27:48.617409 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.596129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:37619 (local address 127.1.8.65:44351)
0806 22:27:48.608202 (+ 12073us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.608209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.610583 (+ 2374us) server_negotiation.cc:407] Connection header received
0806 22:27:48.610810 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.610815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.610894 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.611004 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:48.612040 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.612971 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.615353 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.616409 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.616558 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.616666 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.616954 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.617102 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11903,"thread_start_us":15582,"threads_started":1}
I20260806 22:27:48.617449 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.595939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44351 (local address 127.1.8.67:37619)
0806 22:27:48.610457 (+ 14518us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.610484 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.610661 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.611422 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.611429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.611463 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:48.611844 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.611862 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.613093 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.613098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.615184 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.615202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.616591 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.616597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.616795 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.616821 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.616927 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.617061 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14340,"spinlock_wait_cycles":5888,"thread_start_us":15241,"threads_started":1}
I20260806 22:27:48.618372 14265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.595939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:39267 (local address 127.1.8.67:40593)
0806 22:27:48.611468 (+ 15529us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.611486 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.611610 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.612514 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.612519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.612545 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:27:48.612904 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.612914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.614137 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.614148 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.615818 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.615834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.617410 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.617416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.617608 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.617641 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.617700 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.617816 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15393,"mutex_wait_us":31,"thread_start_us":14810,"threads_started":1}
I20260806 22:27:48.618287 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcd3810d11f459bb41654abe28502c7" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bb8a01de1643f5a9abec5a4c101474" is_pre_election: true
I20260806 22:27:48.618685 13614 raft_consensus.cc:2454] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 0.
I20260806 22:27:48.618919 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.596123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:40593 (local address 127.1.8.66:39267)
0806 22:27:48.612195 (+ 16072us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.612201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.612219 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:27:48.612284 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.612290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.612357 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.612461 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:27:48.613075 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.613998 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.615981 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.617182 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.618195 (+ 1013us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.618308 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.618364 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.618612 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15893,"thread_start_us":15787,"threads_started":1}
I20260806 22:27:48.619297 13953 leader_election.cc:302] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:48.619477 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcd3810d11f459bb41654abe28502c7" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306" is_pre_election: true
I20260806 22:27:48.619787 13829 raft_consensus.cc:2454] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 0.
I20260806 22:27:48.619777 14207 raft_consensus.cc:2781] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:48.619928 14207 raft_consensus.cc:482] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:48.620061 14207 raft_consensus.cc:3037] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.622483 14207 raft_consensus.cc:504] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.623126 14207 leader_election.cc:288] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 election: Requested vote from peers f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351), bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267)
I20260806 22:27:48.623421 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcd3810d11f459bb41654abe28502c7" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bb8a01de1643f5a9abec5a4c101474"
I20260806 22:27:48.623554 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcd3810d11f459bb41654abe28502c7" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306"
I20260806 22:27:48.623636 13614 raft_consensus.cc:3037] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.623797 13829 raft_consensus.cc:3037] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:48.626636 13829 raft_consensus.cc:2454] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 1.
I20260806 22:27:48.626669 13614 raft_consensus.cc:2454] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 1.
I20260806 22:27:48.627038 13955 leader_election.cc:302] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, bf9e6ca72f5541fd81e717ae560e6306; no voters:
I20260806 22:27:48.627386 14207 raft_consensus.cc:2781] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:48.627838 14207 raft_consensus.cc:686] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 LEADER]: Becoming Leader. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Running, Role: LEADER
I20260806 22:27:48.628253 14207 consensus_queue.cc:237] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.630204 14271 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: TxnStatusTablet state changed. Reason: New leader 5af5fe54295a44b28e6d8b15d23b9d57. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.630657 14271 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:27:48.631485 14273 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:27:48.631716 14273 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:27:48.632186 13348 catalog_manager.cc:5495] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } health_report { overall_health: HEALTHY } } }
I20260806 22:27:48.669365 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.662695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.1.8.67:43587)
0806 22:27:48.663347 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.663351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.663371 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:27:48.663445 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.663448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.663522 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.663665 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:48.664156 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.665009 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.666935 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.667391 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.667487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.668079 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.668714 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.668820 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.669024 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.669090 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":383,"threads_started":1}
I20260806 22:27:48.669464 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.662688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43587 (local address 127.0.0.1:49860)
0806 22:27:48.662953 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.662977 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.663148 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.663721 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.663727 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.663741 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:27:48.663946 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.663958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.665152 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.665158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.666793 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.666812 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.667547 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.667554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.667787 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.667865 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.668829 (+ 964us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.668864 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.668971 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.669135 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:27:48.674031 13829 raft_consensus.cc:1261] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Refusing update from remote peer 5af5fe54295a44b28e6d8b15d23b9d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.674400 13614 raft_consensus.cc:1261] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Refusing update from remote peer 5af5fe54295a44b28e6d8b15d23b9d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.674835 14271 consensus_queue.cc:1035] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.675143 14207 consensus_queue.cc:1035] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.681742 14206 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: TxnStatusTablet state changed. Reason: New leader 5af5fe54295a44b28e6d8b15d23b9d57. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.682127 14206 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.684935 14205 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: TxnStatusTablet state changed. Reason: New leader 5af5fe54295a44b28e6d8b15d23b9d57. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.685261 14205 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.686127 14271 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.686407 14271 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:27:48.689361 14206 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.689659 14206 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P f5bb8a01de1643f5a9abec5a4c101474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.692700 14281 mvcc.cc:205] Tried to move back new op lower bound from 7315682380481884160 to 7315682380305330176. Current Snapshot: MvccSnapshot[applied={T|T < 7315682380481884160}]
I20260806 22:27:48.694669 14271 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.695042 14271 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P 5af5fe54295a44b28e6d8b15d23b9d57: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:27:48.695869 14205 tablet_replica.cc:439] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } } }
I20260806 22:27:48.696134 14205 tablet_replica.cc:442] T 2dcd3810d11f459bb41654abe28502c7 P bf9e6ca72f5541fd81e717ae560e6306: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:27:48.709121 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.703506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:57831 (local address 127.1.8.67:43587)
0806 22:27:48.703662 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.703667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.704258 (+ 591us) server_negotiation.cc:407] Connection header received
0806 22:27:48.704477 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.704487 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.704551 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.704645 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:48.705118 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.705875 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.707215 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.707685 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.707829 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.708295 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.708397 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.708496 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.708653 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.708748 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:27:48.709116 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.703520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:43587 (local address 127.1.8.65:57831)
0806 22:27:48.704157 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.704180 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.704344 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.704662 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.704667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.704679 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:27:48.704970 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.704978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.706028 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.706034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.707075 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.707087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.707846 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.707852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.708001 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.708054 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.708498 (+ 444us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.708519 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.708626 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.708769 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":319,"threads_started":1}
I20260806 22:27:48.718919 14071 raft_consensus.cc:1261] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.718930 13829 raft_consensus.cc:1261] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:48.719475 14213 consensus_queue.cc:1035] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.719782 14206 consensus_queue.cc:1035] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:48.757900 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.750192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:51523 (local address 127.1.8.65:44351)
0806 22:27:48.750393 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.750401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.751100 (+ 699us) server_negotiation.cc:407] Connection header received
0806 22:27:48.751308 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.751314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.751406 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.751496 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:48.752446 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.753572 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.755917 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.756591 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.756750 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.757305 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.757414 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.757489 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.757642 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.757703 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:27:48.758448 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.750194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44351 (local address 127.1.8.67:51523)
0806 22:27:48.750967 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:48.750995 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:48.751163 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:48.751735 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:48.751740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:48.751753 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:27:48.752259 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.752273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.753762 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.753767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.755726 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:48.755792 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.756819 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.756826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:48.757016 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.757100 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:27:48.757520 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:27:48.757533 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:27:48.757633 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:27:48.757773 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":408,"threads_started":1}
I20260806 22:27:48.793311 13344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:48.793601 13344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:48.796959 13344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Sending DeleteTablet for 3 replicas of tablet ea8bd6b60be34c4b9ddb3250630ab5c0
I20260806 22:27:48.797685 13577 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42498
I20260806 22:27:48.798151 13796 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55708
I20260806 22:27:48.798346 14037 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8bd6b60be34c4b9ddb3250630ab5c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:48 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49852
I20260806 22:27:48.803984 14309 tablet_replica.cc:330] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: stopping tablet replica
I20260806 22:27:48.804420 14309 raft_consensus.cc:2229] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:48.805751 14309 raft_consensus.cc:2258] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.809331 14312 tablet_replica.cc:330] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: stopping tablet replica
I20260806 22:27:48.809681 14312 raft_consensus.cc:2229] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:48.810109 14312 raft_consensus.cc:2258] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.811908 14309 ts_tablet_manager.cc:1905] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.813047 14307 tablet_replica.cc:330] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: stopping tablet replica
I20260806 22:27:48.813269 14307 raft_consensus.cc:2229] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:48.814123 14307 raft_consensus.cc:2258] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:48.815210 14307 ts_tablet_manager.cc:1905] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.820173 14312 ts_tablet_manager.cc:1905] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:48.822304 14307 ts_tablet_manager.cc:1918] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:48.822438 14307 log.cc:1196] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/ea8bd6b60be34c4b9ddb3250630ab5c0
I20260806 22:27:48.822922 14307 ts_tablet_manager.cc:1939] T ea8bd6b60be34c4b9ddb3250630ab5c0 P f5bb8a01de1643f5a9abec5a4c101474: Deleting consensus metadata
I20260806 22:27:48.824034 13323 catalog_manager.cc:4841] TS f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351): tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (table test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]) successfully deleted
I20260806 22:27:48.819228 14309 ts_tablet_manager.cc:1918] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:48.827044 14309 log.cc:1196] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/ea8bd6b60be34c4b9ddb3250630ab5c0
I20260806 22:27:48.827622 14309 ts_tablet_manager.cc:1939] T ea8bd6b60be34c4b9ddb3250630ab5c0 P bf9e6ca72f5541fd81e717ae560e6306: Deleting consensus metadata
I20260806 22:27:48.828227 14312 ts_tablet_manager.cc:1918] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:48.828365 14312 log.cc:1196] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/ea8bd6b60be34c4b9ddb3250630ab5c0
I20260806 22:27:48.828802 14312 ts_tablet_manager.cc:1939] T ea8bd6b60be34c4b9ddb3250630ab5c0 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting consensus metadata
I20260806 22:27:48.829099 13327 catalog_manager.cc:4841] TS bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267): tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (table test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]) successfully deleted
I20260806 22:27:48.829952 13328 catalog_manager.cc:4841] TS 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587): tablet ea8bd6b60be34c4b9ddb3250630ab5c0 (table test-table [id=a6f72c8d14ec4bc08dee772c6a0a23de]) successfully deleted
Aug 06 22:27:48 dist-test-slave-7vjv krb5kdc[13275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:27:48 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:48 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:27:48.881961 14260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:48.870808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.1.8.126:43921)
0806 22:27:48.871029 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.871035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.871490 (+ 455us) server_negotiation.cc:407] Connection header received
0806 22:27:48.871659 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.871665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.871753 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.871879 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:48.873420 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.874605 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.875970 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.876635 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.876787 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.881675 (+ 4888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52978: BlockingRecv error: recv got EOF from 127.0.0.1:52978 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 22:27:48 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM
I20260806 22:27:48.899709 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:48.884725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.1.8.126:43921)
0806 22:27:48.884895 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:48.884901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:48.885368 (+ 467us) server_negotiation.cc:407] Connection header received
0806 22:27:48.885520 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:48.885528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:48.885589 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:48.885666 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:48.887203 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.887939 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.889646 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:48.890042 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:48.890140 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:48.894313 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:48.894317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:48.894321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:48.894369 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:48.897103 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:48.897755 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:48.897762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:48.897765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:48.897925 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:48.898307 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:48.898312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:48.898317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:48.898578 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:48.898865 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:48.899123 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:27:48.899340 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:27:48.903155 13344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:27:48.903793 13344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:48.910843 13577 tablet_service.cc:1467] Processing CreateTablet for tablet e840c9756d284f7abef3982830bc24df (DEFAULT_TABLE table=test-table [id=84e5893f26174d7aa4ca738668a8f590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.911191 13796 tablet_service.cc:1467] Processing CreateTablet for tablet e840c9756d284f7abef3982830bc24df (DEFAULT_TABLE table=test-table [id=84e5893f26174d7aa4ca738668a8f590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.911307 13577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e840c9756d284f7abef3982830bc24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.911490 14037 tablet_service.cc:1467] Processing CreateTablet for tablet e840c9756d284f7abef3982830bc24df (DEFAULT_TABLE table=test-table [id=84e5893f26174d7aa4ca738668a8f590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:48.911628 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e840c9756d284f7abef3982830bc24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.911991 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e840c9756d284f7abef3982830bc24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:48.918138 14196 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Bootstrap starting.
I20260806 22:27:48.918291 14200 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Bootstrap starting.
I20260806 22:27:48.919793 14200 tablet_bootstrap.cc:654] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.919890 14196 tablet_bootstrap.cc:654] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.933076 14196 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: No bootstrap required, opened a new log
I20260806 22:27:48.933243 14196 ts_tablet_manager.cc:1397] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260806 22:27:48.933588 14336 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Bootstrap starting.
I20260806 22:27:48.934062 14196 raft_consensus.cc:348] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.934324 14196 raft_consensus.cc:374] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.934446 14196 raft_consensus.cc:729] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.934729 14196 consensus_queue.cc:260] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.935268 14336 tablet_bootstrap.cc:654] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:48.935979 14196 ts_tablet_manager.cc:1428] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:27:48.939163 14336 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: No bootstrap required, opened a new log
I20260806 22:27:48.939332 14336 ts_tablet_manager.cc:1397] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 22:27:48.940302 14200 tablet_bootstrap.cc:492] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: No bootstrap required, opened a new log
I20260806 22:27:48.940271 14336 raft_consensus.cc:348] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.940457 14200 ts_tablet_manager.cc:1397] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.008s
I20260806 22:27:48.940549 14336 raft_consensus.cc:374] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.940691 14336 raft_consensus.cc:729] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.940981 14336 consensus_queue.cc:260] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.941352 14200 raft_consensus.cc:348] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.941529 14336 ts_tablet_manager.cc:1428] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:27:48.941602 14200 raft_consensus.cc:374] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:48.941684 14200 raft_consensus.cc:729] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e6ca72f5541fd81e717ae560e6306, State: Initialized, Role: FOLLOWER
I20260806 22:27:48.941948 14200 consensus_queue.cc:260] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:48.942535 14200 ts_tablet_manager.cc:1428] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:27:49.015682 14287 raft_consensus.cc:482] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:49.015974 14287 raft_consensus.cc:504] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.017048 14287 leader_election.cc:288] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351), bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267)
I20260806 22:27:49.017686 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e840c9756d284f7abef3982830bc24df" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bb8a01de1643f5a9abec5a4c101474" is_pre_election: true
I20260806 22:27:49.017705 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e840c9756d284f7abef3982830bc24df" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306" is_pre_election: true
I20260806 22:27:49.017962 13614 raft_consensus.cc:2454] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 0.
I20260806 22:27:49.018038 13828 raft_consensus.cc:2454] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 0.
I20260806 22:27:49.018419 13953 leader_election.cc:302] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:49.018726 14287 raft_consensus.cc:2781] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:49.018875 14287 raft_consensus.cc:482] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:49.018982 14287 raft_consensus.cc:3037] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.022511 14287 raft_consensus.cc:504] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.023253 14287 leader_election.cc:288] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 election: Requested vote from peers f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351), bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267)
I20260806 22:27:49.023764 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e840c9756d284f7abef3982830bc24df" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306"
I20260806 22:27:49.023797 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e840c9756d284f7abef3982830bc24df" candidate_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bb8a01de1643f5a9abec5a4c101474"
I20260806 22:27:49.023978 13828 raft_consensus.cc:3037] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.024044 13614 raft_consensus.cc:3037] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.026034 13828 raft_consensus.cc:2454] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 1.
I20260806 22:27:49.026503 13955 leader_election.cc:302] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, bf9e6ca72f5541fd81e717ae560e6306; no voters:
I20260806 22:27:49.026836 14287 raft_consensus.cc:2781] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:49.027076 14287 raft_consensus.cc:686] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 LEADER]: Becoming Leader. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Running, Role: LEADER
I20260806 22:27:49.027650 13614 raft_consensus.cc:2454] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af5fe54295a44b28e6d8b15d23b9d57 in term 1.
I20260806 22:27:49.027473 14287 consensus_queue.cc:237] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.031075 13344 catalog_manager.cc:5495] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } health_report { overall_health: HEALTHY } } }
I20260806 22:27:49.055508 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:49.050388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.1.8.67:43587)
0806 22:27:49.050601 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:49.050605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:49.050621 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:27:49.050697 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:49.050701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:49.050800 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:49.050914 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:49.051335 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.052255 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.053556 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.054067 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.054189 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:49.054731 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:49.054831 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:49.054954 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:49.055122 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:27:49.055198 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:27:49.059350 13828 raft_consensus.cc:1261] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Refusing update from remote peer 5af5fe54295a44b28e6d8b15d23b9d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:49.059440 13614 raft_consensus.cc:1261] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Refusing update from remote peer 5af5fe54295a44b28e6d8b15d23b9d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:49.060058 14287 consensus_queue.cc:1035] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:49.060364 14207 consensus_queue.cc:1035] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:49.081163 13344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:49.081449 13344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:49.089663 13344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Sending DeleteTablet for 3 replicas of tablet e840c9756d284f7abef3982830bc24df
I20260806 22:27:49.090848 13577 tablet_service.cc:1514] Processing DeleteTablet for tablet e840c9756d284f7abef3982830bc24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42498
I20260806 22:27:49.090929 13796 tablet_service.cc:1514] Processing DeleteTablet for tablet e840c9756d284f7abef3982830bc24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55708
I20260806 22:27:49.091274 14037 tablet_service.cc:1514] Processing DeleteTablet for tablet e840c9756d284f7abef3982830bc24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49852
I20260806 22:27:49.091641 13795 tablet_service.cc:1514] Processing DeleteTablet for tablet e840c9756d284f7abef3982830bc24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55708
I20260806 22:27:49.091971 14307 tablet_replica.cc:330] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: stopping tablet replica
I20260806 22:27:49.092218 14307 raft_consensus.cc:2229] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:49.092473 14307 raft_consensus.cc:2258] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.092799 14309 tablet_replica.cc:330] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: stopping tablet replica
I20260806 22:27:49.093024 14309 raft_consensus.cc:2229] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:49.093253 14309 raft_consensus.cc:2258] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.093568 14312 tablet_replica.cc:330] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: stopping tablet replica
I20260806 22:27:49.093396 13348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:27:49.093809 14312 raft_consensus.cc:2229] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 LEADER]: Raft consensus shutting down.
W20260806 22:27:49.094099 13348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:27:49.094360 14312 raft_consensus.cc:2258] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.096150 14312 ts_tablet_manager.cc:1905] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.100942 14309 ts_tablet_manager.cc:1905] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.106643 14312 ts_tablet_manager.cc:1918] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:49.106890 14312 log.cc:1196] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/e840c9756d284f7abef3982830bc24df
I20260806 22:27:49.107656 13577 tablet_service.cc:1467] Processing CreateTablet for tablet 271f5b691a4047dba8badb14813eef31 (DEFAULT_TABLE table=test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:49.107834 14312 ts_tablet_manager.cc:1939] T e840c9756d284f7abef3982830bc24df P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting consensus metadata
I20260806 22:27:49.108139 13577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271f5b691a4047dba8badb14813eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:49.110622 14307 ts_tablet_manager.cc:1905] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.112982 13328 catalog_manager.cc:4841] TS 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587): tablet e840c9756d284f7abef3982830bc24df (table test-table [id=84e5893f26174d7aa4ca738668a8f590]) successfully deleted
I20260806 22:27:49.113903 14037 tablet_service.cc:1467] Processing CreateTablet for tablet 271f5b691a4047dba8badb14813eef31 (DEFAULT_TABLE table=test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:49.114377 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271f5b691a4047dba8badb14813eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:49.117367 14309 ts_tablet_manager.cc:1918] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:49.117591 14309 log.cc:1196] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/e840c9756d284f7abef3982830bc24df
I20260806 22:27:49.118070 14309 ts_tablet_manager.cc:1939] T e840c9756d284f7abef3982830bc24df P bf9e6ca72f5541fd81e717ae560e6306: Deleting consensus metadata
I20260806 22:27:49.119549 13795 tablet_service.cc:1467] Processing CreateTablet for tablet 271f5b691a4047dba8badb14813eef31 (DEFAULT_TABLE table=test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:27:49.119993 13795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271f5b691a4047dba8badb14813eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:27:49.119972 13327 catalog_manager.cc:4841] TS bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267): tablet e840c9756d284f7abef3982830bc24df (table test-table [id=84e5893f26174d7aa4ca738668a8f590]) successfully deleted
W20260806 22:27:49.120698 13327 catalog_manager.cc:4808] TS bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267): delete failed for tablet e840c9756d284f7abef3982830bc24df because the tablet was not found. No further retry: Not found: Tablet not found: e840c9756d284f7abef3982830bc24df
I20260806 22:27:49.121531 14307 ts_tablet_manager.cc:1918] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:27:49.121691 14307 log.cc:1196] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/e840c9756d284f7abef3982830bc24df
I20260806 22:27:49.122154 14307 ts_tablet_manager.cc:1939] T e840c9756d284f7abef3982830bc24df P f5bb8a01de1643f5a9abec5a4c101474: Deleting consensus metadata
I20260806 22:27:49.122290 14196 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Bootstrap starting.
I20260806 22:27:49.123978 14196 tablet_bootstrap.cc:654] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:49.125217 14336 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Bootstrap starting.
I20260806 22:27:49.126034 13323 catalog_manager.cc:4841] TS f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351): tablet e840c9756d284f7abef3982830bc24df (table test-table [id=84e5893f26174d7aa4ca738668a8f590]) successfully deleted
I20260806 22:27:49.126964 14336 tablet_bootstrap.cc:654] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:49.128010 14200 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Bootstrap starting.
I20260806 22:27:49.130168 14200 tablet_bootstrap.cc:654] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:49.135455 14200 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: No bootstrap required, opened a new log
I20260806 22:27:49.135628 14200 ts_tablet_manager.cc:1397] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 22:27:49.136140 14336 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: No bootstrap required, opened a new log
I20260806 22:27:49.136160 14196 tablet_bootstrap.cc:492] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: No bootstrap required, opened a new log
I20260806 22:27:49.136328 14336 ts_tablet_manager.cc:1397] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260806 22:27:49.136323 14196 ts_tablet_manager.cc:1397] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260806 22:27:49.137266 14196 raft_consensus.cc:348] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.137259 14336 raft_consensus.cc:348] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.137534 14336 raft_consensus.cc:374] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:49.137569 14196 raft_consensus.cc:374] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:49.137650 14336 raft_consensus.cc:729] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5fe54295a44b28e6d8b15d23b9d57, State: Initialized, Role: FOLLOWER
I20260806 22:27:49.137691 14196 raft_consensus.cc:729] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Initialized, Role: FOLLOWER
I20260806 22:27:49.137621 14200 raft_consensus.cc:348] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.137872 14200 raft_consensus.cc:374] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:49.137939 14200 raft_consensus.cc:729] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e6ca72f5541fd81e717ae560e6306, State: Initialized, Role: FOLLOWER
I20260806 22:27:49.137959 14336 consensus_queue.cc:260] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.137982 14196 consensus_queue.cc:260] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.138494 14196 ts_tablet_manager.cc:1428] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:27:49.138391 14200 consensus_queue.cc:260] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.140528 14200 ts_tablet_manager.cc:1428] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:27:49.140519 14336 ts_tablet_manager.cc:1428] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 22:27:49.533320 14213 raft_consensus.cc:482] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:27:49.533596 14213 raft_consensus.cc:504] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.534334 14213 leader_election.cc:288] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:49.534953 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271f5b691a4047dba8badb14813eef31" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306" is_pre_election: true
I20260806 22:27:49.534984 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271f5b691a4047dba8badb14813eef31" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" is_pre_election: true
I20260806 22:27:49.535254 14071 raft_consensus.cc:2454] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:49.535270 13828 raft_consensus.cc:2454] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 0.
I20260806 22:27:49.535825 13484 leader_election.cc:302] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:49.536160 14213 raft_consensus.cc:2781] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:27:49.536289 14213 raft_consensus.cc:482] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:27:49.536430 14213 raft_consensus.cc:3037] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.538661 14213 raft_consensus.cc:504] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.539268 14213 leader_election.cc:288] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Requested vote from peers bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267), 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587)
I20260806 22:27:49.539855 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271f5b691a4047dba8badb14813eef31" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af5fe54295a44b28e6d8b15d23b9d57"
I20260806 22:27:49.539856 13828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271f5b691a4047dba8badb14813eef31" candidate_uuid: "f5bb8a01de1643f5a9abec5a4c101474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf9e6ca72f5541fd81e717ae560e6306"
I20260806 22:27:49.540091 13828 raft_consensus.cc:3037] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.540084 14071 raft_consensus.cc:3037] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:49.542161 14071 raft_consensus.cc:2454] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:49.542200 13828 raft_consensus.cc:2454] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bb8a01de1643f5a9abec5a4c101474 in term 1.
I20260806 22:27:49.542658 13484 leader_election.cc:302] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af5fe54295a44b28e6d8b15d23b9d57, f5bb8a01de1643f5a9abec5a4c101474; no voters:
I20260806 22:27:49.542982 14213 raft_consensus.cc:2781] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:49.543354 14213 raft_consensus.cc:686] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Becoming Leader. State: Replica: f5bb8a01de1643f5a9abec5a4c101474, State: Running, Role: LEADER
I20260806 22:27:49.543629 14213 consensus_queue.cc:237] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } }
I20260806 22:27:49.546241 13344 catalog_manager.cc:5495] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "f5bb8a01de1643f5a9abec5a4c101474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5bb8a01de1643f5a9abec5a4c101474" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 } health_report { overall_health: UNKNOWN } } }
I20260806 22:27:49.587515 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:49.581993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.1.8.65:44351)
0806 22:27:49.582510 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:49.582514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:49.582594 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:27:49.582737 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:49.582741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:49.582802 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:49.582881 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:27:49.583626 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.584568 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.585907 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.586350 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.586457 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:49.586805 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:27:49.586936 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:27:49.587014 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:49.587145 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:27:49.587209 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":302,"threads_started":1}
I20260806 22:27:49.596907 13828 raft_consensus.cc:1261] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:49.596907 14071 raft_consensus.cc:1261] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Refusing update from remote peer f5bb8a01de1643f5a9abec5a4c101474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:27:49.597527 14351 consensus_queue.cc:1035] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf9e6ca72f5541fd81e717ae560e6306" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:49.597867 14213 consensus_queue.cc:1035] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af5fe54295a44b28e6d8b15d23b9d57" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:27:49.668011 13348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:49.668292 13348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:27:49.671097 13348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb9fb1cf75f44fd7b9dc97bcb92e9f93: Sending DeleteTablet for 3 replicas of tablet 271f5b691a4047dba8badb14813eef31
I20260806 22:27:49.671885 13795 tablet_service.cc:1514] Processing DeleteTablet for tablet 271f5b691a4047dba8badb14813eef31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55708
I20260806 22:27:49.671991 14037 tablet_service.cc:1514] Processing DeleteTablet for tablet 271f5b691a4047dba8badb14813eef31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49852
I20260806 22:27:49.672065 13577 tablet_service.cc:1514] Processing DeleteTablet for tablet 271f5b691a4047dba8badb14813eef31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:27:49 UTC) from {username='oryx', principal='oryx/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42498
I20260806 22:27:49.672597 14368 tablet_replica.cc:330] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: stopping tablet replica
I20260806 22:27:49.672832 14372 tablet_replica.cc:330] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: stopping tablet replica
I20260806 22:27:49.672816 14368 raft_consensus.cc:2229] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:49.673038 14372 raft_consensus.cc:2229] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:27:49.673146 14368 raft_consensus.cc:2258] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.673166 14374 tablet_replica.cc:330] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: stopping tablet replica
I20260806 22:27:49.673432 14374 raft_consensus.cc:2229] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:27:49.673636 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 13454
I20260806 22:27:49.674070 14374 raft_consensus.cc:2258] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.675159 14374 ts_tablet_manager.cc:1905] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.675426 14372 raft_consensus.cc:2258] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:27:49.676033 14368 ts_tablet_manager.cc:1905] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.676505 14372 ts_tablet_manager.cc:1905] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:27:49.682400 14374 ts_tablet_manager.cc:1918] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:49.682595 14374 log.cc:1196] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/271f5b691a4047dba8badb14813eef31
I20260806 22:27:49.682976 14368 ts_tablet_manager.cc:1918] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:49.683061 14372 ts_tablet_manager.cc:1918] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:27:49.683122 14368 log.cc:1196] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/271f5b691a4047dba8badb14813eef31
I20260806 22:27:49.683198 14372 log.cc:1196] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/271f5b691a4047dba8badb14813eef31
I20260806 22:27:49.683539 14374 ts_tablet_manager.cc:1939] T 271f5b691a4047dba8badb14813eef31 P f5bb8a01de1643f5a9abec5a4c101474: Deleting consensus metadata
I20260806 22:27:49.683671 14368 ts_tablet_manager.cc:1939] T 271f5b691a4047dba8badb14813eef31 P bf9e6ca72f5541fd81e717ae560e6306: Deleting consensus metadata
I20260806 22:27:49.683684 14372 ts_tablet_manager.cc:1939] T 271f5b691a4047dba8badb14813eef31 P 5af5fe54295a44b28e6d8b15d23b9d57: Deleting consensus metadata
I20260806 22:27:49.684785 13323 catalog_manager.cc:4841] TS f5bb8a01de1643f5a9abec5a4c101474 (127.1.8.65:44351): tablet 271f5b691a4047dba8badb14813eef31 (table test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]) successfully deleted
I20260806 22:27:49.684896 13327 catalog_manager.cc:4841] TS bf9e6ca72f5541fd81e717ae560e6306 (127.1.8.66:39267): tablet 271f5b691a4047dba8badb14813eef31 (table test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]) successfully deleted
I20260806 22:27:49.684785 13328 catalog_manager.cc:4841] TS 5af5fe54295a44b28e6d8b15d23b9d57 (127.1.8.67:43587): tablet 271f5b691a4047dba8badb14813eef31 (table test-table [id=d210f64e30c24ee4b012b51cf7e0d7d1]) successfully deleted
Aug 06 22:27:49 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.65@KRBTEST.COM
I20260806 22:27:49.691612 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:49.675042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.1.8.65:44351)
0806 22:27:49.675230 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:49.675236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:49.675255 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:49.675468 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:49.675473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:49.675555 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:49.675654 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:49.677554 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.678662 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.680494 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.681148 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.681266 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:49.686446 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:49.686461 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:49.686477 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:49.686517 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:49.689254 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:49.689891 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:49.689898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:49.689900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:49.690026 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:49.690380 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:49.690384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:49.690386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:49.690762 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:49.691017 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:49.691206 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:27:49.691373 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:27:49.692381 13677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:49.820688 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 13454
I20260806 22:27:49.851994 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 13693
Aug 06 22:27:49 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.66@KRBTEST.COM
I20260806 22:27:49.867534 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:49.852806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.1.8.66:39267)
0806 22:27:49.853348 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:49.853353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:49.853370 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:27:49.853426 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:49.853429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:49.853488 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:49.853580 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:49.854801 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.855541 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.856826 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:49.857241 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:49.857334 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:49.861657 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:49.861673 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:49.861690 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:49.861728 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:49.864915 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:49.865513 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:49.865520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:49.865522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:49.865663 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:49.865999 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:49.866003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:49.866005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:49.866472 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:49.866722 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:49.866967 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:27:49.867145 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":273,"threads_started":1}
I20260806 22:27:49.868276 13904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:49.969040 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 13693
I20260806 22:27:49.996940 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 13924
Aug 06 22:27:50 dist-test-slave-7vjv krb5kdc[13275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.8.67@KRBTEST.COM
I20260806 22:27:50.017584 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:49.997885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.1.8.67:43587)
0806 22:27:49.998476 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:49.998480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:49.998516 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:27:49.998581 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:49.998584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:49.998654 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:49.998783 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:50.000899 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:50.001693 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:50.003882 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:50.004625 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:50.004738 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:50.011251 (+ 6513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:50.011269 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:50.011285 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:50.011327 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:50.014495 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:50.015251 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:50.015268 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:50.015270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:50.015418 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:50.015822 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:50.015827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:50.015830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:50.016279 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:50.016607 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:50.016956 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 22:27:50.017208 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":315,"threads_started":1}
I20260806 22:27:50.018301 14146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:50.207577 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 13924
I20260806 22:27:50.245666 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 13300
I20260806 22:27:50.246904 13409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:50.370973 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 13300
2026-08-06T22:27:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6645 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:50 dist-test-slave-7vjv krb5kdc[14391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:50 dist-test-slave-7vjv krb5kdc[14391](info): set up 2 sockets
Aug 06 22:27:50 dist-test-slave-7vjv krb5kdc[14391](info): commencing operation
krb5kdc: starting...
W20260806 22:27:51.713177 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.012s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:51 dist-test-slave-7vjv krb5kdc[14391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:52.013983 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:35823
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:52.362816 14408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:52.363090 14408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:52.363251 14408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:52.369940 14408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:52.370033 14408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:52.370134 14408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:52.370230 14408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:52.370311 14408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:52.379212 14408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:52.380447 14408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:52.381681 14408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:52.393137 14415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:52.393103 14416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:52.393065 14418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:52.394335 14408 server_base.cc:1034] running on GCE node
I20260806 22:27:52.395061 14408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:52.396343 14408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:52.397553 14408 hybrid_clock.cc:648] HybridClock initialized: now 1786055272397511 us; error 31 us; skew 500 ppm
I20260806 22:27:52.397876 14408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:27:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2085 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:27:52 dist-test-slave-7vjv krb5kdc[14430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:27:52 dist-test-slave-7vjv krb5kdc[14430](info): set up 2 sockets
Aug 06 22:27:52 dist-test-slave-7vjv krb5kdc[14430](info): commencing operation
krb5kdc: starting...
W20260806 22:27:53.970027 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.455s user 0.009s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:27:54 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:27:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:27:54.258203 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38487
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:54.621603 14453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:54.621917 14453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:54.622010 14453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:54.629371 14453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:54.629477 14453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:54.629526 14453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:54.629585 14453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:54.629623 14453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:54.639714 14453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:54.641108 14453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:54.642508 14453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:54.656180 14464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:54.656210 14462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:54.656265 14461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:54.657433 14453 server_base.cc:1034] running on GCE node
I20260806 22:27:54.658315 14453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:54.659813 14453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:54.661055 14453 hybrid_clock.cc:648] HybridClock initialized: now 1786055274660976 us; error 89 us; skew 500 ppm
I20260806 22:27:54.661420 14453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:54 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:54.667480 14453 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:54.669113 14453 webserver.cc:460] Webserver started at http://127.1.8.126:41317/ using document root <none> and password file <none>
I20260806 22:27:54.669857 14453 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:54.670010 14453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:54.670342 14453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:54.673521 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "28a54f604c0040a7a7fcc25643340aba"
format_stamp: "Formatted at 2026-08-06 22:27:54 on dist-test-slave-7vjv"
server_key: "cfc4a973ec4f1048c167553d435b8b9d"
server_key_iv: "840a14907520733f1279bf1326ab606e"
server_key_version: "encryptionkey@0"
I20260806 22:27:54.674346 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "28a54f604c0040a7a7fcc25643340aba"
format_stamp: "Formatted at 2026-08-06 22:27:54 on dist-test-slave-7vjv"
server_key: "cfc4a973ec4f1048c167553d435b8b9d"
server_key_iv: "840a14907520733f1279bf1326ab606e"
server_key_version: "encryptionkey@0"
I20260806 22:27:54.679787 14453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 22:27:54.683320 14476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:54.684811 14453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:54.684964 14453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "28a54f604c0040a7a7fcc25643340aba"
format_stamp: "Formatted at 2026-08-06 22:27:54 on dist-test-slave-7vjv"
server_key: "cfc4a973ec4f1048c167553d435b8b9d"
server_key_iv: "840a14907520733f1279bf1326ab606e"
server_key_version: "encryptionkey@0"
I20260806 22:27:54.685148 14453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:54.708818 14453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:54.712822 14453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:54.713078 14453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:54.733667 14453 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38487
I20260806 22:27:54.733713 14555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38487 every 8 connection(s)
I20260806 22:27:54.735193 14453 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:54.737175 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 14453
I20260806 22:27:54.737444 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:27:54.738130 1057 external_mini_cluster.cc:1428] Setting key e5ee8359c6653a62eb4d7f176971a1b7
I20260806 22:27:54.739516 14556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:27:54 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:54.753969 14556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Bootstrap starting.
I20260806 22:27:54.758494 14556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Neither blocks nor log segments found. Creating new log.
I20260806 22:27:54.759886 14556 log.cc:824] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:54.762996 14556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: No bootstrap required, opened a new log
I20260806 22:27:54.763381 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:54.740252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.1.8.126:38487)
0806 22:27:54.741158 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:54.741173 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:54.741226 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:27:54.743216 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:54.743244 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:54.743753 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:54.744225 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:54.745675 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:54.747816 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:54.749174 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:54.750233 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:54.750431 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:54.754751 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:54.754763 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:54.754781 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:54.754839 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:54.759101 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:54.759809 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:54.759819 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:54.759830 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:54.760067 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:54.760488 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:54.760500 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:54.760505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:54.761109 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:54.761480 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:54.761905 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 22:27:54.762331 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":397,"threads_started":1}
I20260806 22:27:54.771145 14556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:54.771548 14556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:27:54.771680 14556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a54f604c0040a7a7fcc25643340aba, State: Initialized, Role: FOLLOWER
I20260806 22:27:54.772294 14556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:54.772516 14556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:54.772616 14556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:54.772751 14556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:27:54.774820 14556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:54.775298 14556 leader_election.cc:302] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28a54f604c0040a7a7fcc25643340aba; no voters:
I20260806 22:27:54.775931 14556 leader_election.cc:288] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:27:54.776029 14566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:27:54.776845 14566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 LEADER]: Becoming Leader. State: Replica: 28a54f604c0040a7a7fcc25643340aba, State: Running, Role: LEADER
I20260806 22:27:54.777503 14566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:54.778255 14556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:54.780608 14568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28a54f604c0040a7a7fcc25643340aba. Latest consensus state: current_term: 1 leader_uuid: "28a54f604c0040a7a7fcc25643340aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } } }
I20260806 22:27:54.780637 14567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28a54f604c0040a7a7fcc25643340aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } } }
I20260806 22:27:54.781060 14568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:54.781067 14567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:54.783627 14576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:54.789037 14576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:54.797363 14576 catalog_manager.cc:1347] Generated new cluster ID: 1f75f2c76d9843f8b0bc0fc6ed855307
I20260806 22:27:54.797472 14576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:54.813287 14576 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:27:54.815136 14576 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:54.832358 14576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Generated new TSK 0
I20260806 22:27:54.833361 14576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:27:54.912415 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:55.259402 14596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:55.259783 14596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:55.259974 14596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:55.267647 14596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:55.267815 14596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:55.268007 14596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:55.278806 14596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:55.280349 14596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:55.281787 14596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:55.296223 14611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:55.296315 14604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:55.296265 14606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:55.297019 14596 server_base.cc:1034] running on GCE node
I20260806 22:27:55.297537 14596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:55.298363 14596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:55.299561 14596 hybrid_clock.cc:648] HybridClock initialized: now 1786055275299522 us; error 26 us; skew 500 ppm
I20260806 22:27:55.299849 14596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:55 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:55.303272 14596 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:27:55.304857 14596 webserver.cc:460] Webserver started at http://127.1.8.65:45601/ using document root <none> and password file <none>
I20260806 22:27:55.305546 14596 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:55.305697 14596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:55.306051 14596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:55.309131 14596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "f0d8f9cbe2f0467e9e3065b7a717da59"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "09ab4367ef11df777a04c7ef0576bb61"
server_key_iv: "8c1e4c6d35291be15cbd051faa45d12a"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.310019 14596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "f0d8f9cbe2f0467e9e3065b7a717da59"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "09ab4367ef11df777a04c7ef0576bb61"
server_key_iv: "8c1e4c6d35291be15cbd051faa45d12a"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.315402 14596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:27:55.318955 14618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.320302 14596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:27:55.320472 14596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "f0d8f9cbe2f0467e9e3065b7a717da59"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "09ab4367ef11df777a04c7ef0576bb61"
server_key_iv: "8c1e4c6d35291be15cbd051faa45d12a"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.320672 14596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:55.335515 14596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:55.339406 14596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:55.339735 14596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:55.340759 14596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:55.342167 14596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:55.342249 14596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.342417 14596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:55.342499 14596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.378924 14596 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:32853
I20260806 22:27:55.378968 14802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:32853 every 8 connection(s)
I20260806 22:27:55.380472 14596 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:55.382171 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 14596
I20260806 22:27:55.382457 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:27:55.383208 1057 external_mini_cluster.cc:1428] Setting key 2381694dc53bf55d502eedc52f5c914b
Aug 06 22:27:55 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:55.403682 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:55.383287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:37795 (local address 127.1.8.126:38487)
0806 22:27:55.384302 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:55.384308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:55.385061 (+ 753us) server_negotiation.cc:407] Connection header received
0806 22:27:55.387648 (+ 2587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:55.387652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:55.387714 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:55.387807 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:55.389920 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.390623 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.392983 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.393397 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.393508 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:55.397576 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:55.397580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:55.397583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:55.397614 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:55.400706 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:55.401395 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:55.401401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:55.401404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:55.401529 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:55.401949 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:55.401954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:55.401957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:55.402217 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:55.402383 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:55.403093 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 22:27:55.403315 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":773,"threads_started":1}
I20260806 22:27:55.404675 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:55.383805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:37795)
0806 22:27:55.384915 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:55.384959 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:55.387350 (+ 2391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:55.388070 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:55.388082 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:55.388928 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:55.389721 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:55.389747 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.390777 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.390784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:55.392822 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:55.392840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.393627 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.393636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:55.393955 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:55.394677 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:55.394709 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:55.397358 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:55.400847 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:55.400856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:55.400875 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:55.401274 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:55.401636 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:55.401640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:55.401644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:55.401824 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:55.402343 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:55.402353 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:55.402721 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 22:27:55.402974 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 22:27:55.403486 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":692,"threads_started":1}
I20260806 22:27:55.406314 14804 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38487
I20260806 22:27:55.406654 14804 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:55.407354 14804 heartbeater.cc:511] Master 127.1.8.126:38487 requested a full tablet report, sending...
I20260806 22:27:55.409507 14502 ts_manager.cc:194] Registered new tserver with Master: f0d8f9cbe2f0467e9e3065b7a717da59 (127.1.8.65:32853)
I20260806 22:27:55.410935 14502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:37795
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:27:55.447118 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:55.781913 14812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:55.782182 14812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:55.782282 14812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:55.789708 14812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:55.789815 14812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:55.790006 14812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:55.799417 14812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:55.800704 14812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:55.802028 14812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:55.815557 14819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:55.815435 14825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:55.815462 14820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:55.816645 14812 server_base.cc:1034] running on GCE node
I20260806 22:27:55.817222 14812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:55.818027 14812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:55.819267 14812 hybrid_clock.cc:648] HybridClock initialized: now 1786055275819212 us; error 64 us; skew 500 ppm
I20260806 22:27:55.819564 14812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:55 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:55.822697 14812 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:27:55.824232 14812 webserver.cc:460] Webserver started at http://127.1.8.66:38205/ using document root <none> and password file <none>
I20260806 22:27:55.824893 14812 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:55.825040 14812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:55.825403 14812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:55.828274 14812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "76a3ef075d734ebbaefe10a584b470cd"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "99c8195606276a57809a4862fd3c2073"
server_key_iv: "5a11bec58a01a2b58addb31b01e0bec5"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.829075 14812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "76a3ef075d734ebbaefe10a584b470cd"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "99c8195606276a57809a4862fd3c2073"
server_key_iv: "5a11bec58a01a2b58addb31b01e0bec5"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.833998 14812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:27:55.837218 14835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.838443 14812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:55.838613 14812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "76a3ef075d734ebbaefe10a584b470cd"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "99c8195606276a57809a4862fd3c2073"
server_key_iv: "5a11bec58a01a2b58addb31b01e0bec5"
server_key_version: "encryptionkey@0"
I20260806 22:27:55.838798 14812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:55.860285 14812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:55.863993 14812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:55.864311 14812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:55.865254 14812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:55.866595 14812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:55.866709 14812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.866847 14812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:55.866916 14812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:55.901501 14812 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39243
I20260806 22:27:55.901585 15007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39243 every 8 connection(s)
I20260806 22:27:55.902763 14812 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:55.912719 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 14812
I20260806 22:27:55.912930 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:27:55.913360 1057 external_mini_cluster.cc:1428] Setting key b3e2337c2c0d407daab06248d7160a59
Aug 06 22:27:55 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:55.923408 15011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:55.905657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:55459 (local address 127.1.8.126:38487)
0806 22:27:55.906074 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:55.906078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:55.907061 (+ 983us) server_negotiation.cc:407] Connection header received
0806 22:27:55.908931 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:55.908937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:55.909030 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:55.909153 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:55.910644 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.911386 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.912990 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.913438 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.913592 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:55.917634 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:55.917639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:55.917642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:55.917672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:55.920274 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:55.921190 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:55.921193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:55.921195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:55.921314 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:55.921785 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:55.921788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:55.921790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:55.922022 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:55.922145 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:55.922964 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 22:27:55.923129 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":219,"threads_started":1}
I20260806 22:27:55.925009 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:55.906175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:55459)
0806 22:27:55.906898 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:55.906944 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:55.908672 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:55.909380 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:55.909388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:55.909930 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:55.910467 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:55.910484 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.911568 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.911574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:55.912859 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:55.912872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:55.913678 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:55.913686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:55.913908 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:55.914583 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:55.914612 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:55.917409 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:55.920403 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:55.920414 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:55.920440 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:55.921075 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:55.921413 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:55.921427 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:55.921434 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:55.921677 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:55.922137 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:55.922147 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:55.922585 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 22:27:55.922881 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 22:27:55.923447 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":280,"threads_started":1}
I20260806 22:27:55.926476 15008 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38487
I20260806 22:27:55.926783 15008 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:55.927477 15008 heartbeater.cc:511] Master 127.1.8.126:38487 requested a full tablet report, sending...
I20260806 22:27:55.928956 14502 ts_manager.cc:194] Registered new tserver with Master: 76a3ef075d734ebbaefe10a584b470cd (127.1.8.66:39243)
I20260806 22:27:55.929754 14502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:55459
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:27:55.975522 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:56.309657 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:56.309952 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:56.310055 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:56.317436 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:56.317548 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:56.317672 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:56.327761 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:56.329227 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:56.330489 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:56.346208 15025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:56.346267 15027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:56.346496 15030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:56.347966 15017 server_base.cc:1034] running on GCE node
I20260806 22:27:56.348719 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:56.350406 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:56.351629 15017 hybrid_clock.cc:648] HybridClock initialized: now 1786055276351580 us; error 58 us; skew 500 ppm
I20260806 22:27:56.351950 15017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:56 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:56.355319 15017 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:27:56.356828 15017 webserver.cc:460] Webserver started at http://127.1.8.67:39995/ using document root <none> and password file <none>
I20260806 22:27:56.357532 15017 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:56.357682 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:56.358006 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:27:56.361034 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "27f189ffa1524ce081c4ba7b59bb9eb4"
format_stamp: "Formatted at 2026-08-06 22:27:56 on dist-test-slave-7vjv"
server_key: "da7622e42a957adbb257a29a4096d805"
server_key_iv: "94fcdddf13febb907cd8ecb4d3d6f347"
server_key_version: "encryptionkey@0"
I20260806 22:27:56.361832 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "27f189ffa1524ce081c4ba7b59bb9eb4"
format_stamp: "Formatted at 2026-08-06 22:27:56 on dist-test-slave-7vjv"
server_key: "da7622e42a957adbb257a29a4096d805"
server_key_iv: "94fcdddf13febb907cd8ecb4d3d6f347"
server_key_version: "encryptionkey@0"
I20260806 22:27:56.366882 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:27:56.370316 15040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:56.371583 15017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 22:27:56.371754 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "27f189ffa1524ce081c4ba7b59bb9eb4"
format_stamp: "Formatted at 2026-08-06 22:27:56 on dist-test-slave-7vjv"
server_key: "da7622e42a957adbb257a29a4096d805"
server_key_iv: "94fcdddf13febb907cd8ecb4d3d6f347"
server_key_version: "encryptionkey@0"
I20260806 22:27:56.371954 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:56.397730 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:56.402088 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:56.402390 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:56.403384 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:56.404970 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:56.405067 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:56.405210 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:56.405279 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:56.413867 14804 heartbeater.cc:503] Master 127.1.8.126:38487 was elected leader, sending a full tablet report...
I20260806 22:27:56.444346 15017 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:35545
I20260806 22:27:56.444460 15233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:35545 every 8 connection(s)
I20260806 22:27:56.445676 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:56.451447 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 15017
I20260806 22:27:56.451665 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:27:56.452261 1057 external_mini_cluster.cc:1428] Setting key f05c08ce00bf50f1987d88b06abcf22f
Aug 06 22:27:56 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:27:56.467262 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:56.448363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:53521 (local address 127.1.8.126:38487)
0806 22:27:56.448831 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:56.448836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:56.449811 (+ 975us) server_negotiation.cc:407] Connection header received
0806 22:27:56.451857 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:56.451863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:56.451984 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:56.452142 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:56.453839 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.455667 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.457227 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.457649 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.457744 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:56.461157 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:56.461161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:56.461164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:56.461195 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:56.463965 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.464701 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.464710 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.464712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.464859 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.465296 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.465299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.465301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.465635 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:56.465813 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:56.466585 (+ 772us) server_negotiation.cc:299] Negotiation successful
0806 22:27:56.466821 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260806 22:27:56.468205 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:56.448927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:53521)
0806 22:27:56.449667 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:56.449704 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:56.451569 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:56.452338 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:56.452345 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:56.452943 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:56.453595 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:56.453630 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.455805 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.455809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:56.457097 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:56.457110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.457837 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.457843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:56.458058 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:56.458686 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:56.458708 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:56.460908 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:27:56.464083 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:56.464089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:56.464139 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:56.464574 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:56.464985 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:27:56.464988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:27:56.464990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:27:56.465182 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:27:56.465774 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:27:56.465779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:27:56.466155 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 22:27:56.466417 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 22:27:56.466940 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":317,"threads_started":1}
I20260806 22:27:56.469496 15234 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38487
I20260806 22:27:56.469766 15234 heartbeater.cc:464] Registering TS with master...
I20260806 22:27:56.470463 15234 heartbeater.cc:511] Master 127.1.8.126:38487 requested a full tablet report, sending...
I20260806 22:27:56.471920 14502 ts_manager.cc:194] Registered new tserver with Master: 27f189ffa1524ce081c4ba7b59bb9eb4 (127.1.8.67:35545)
I20260806 22:27:56.472882 14502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:53521
I20260806 22:27:56.480266 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
Entry for principal oryx/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab.
WARNING: no policy specified for oryx/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
Entry for principal oryx/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab.
WARNING: no policy specified for oryx/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
Entry for principal oryx/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab.
I20260806 22:27:56.656021 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 14596
Aug 06 22:27:56 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:27:56.672437 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:56.657292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.1.8.65:32853)
0806 22:27:56.658032 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:56.658040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:56.658067 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:27:56.658131 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:56.658137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:56.658353 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:56.658489 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:56.659818 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.660881 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.662070 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.662624 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.662720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:56.666975 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:56.666987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:56.667000 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:56.667037 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:56.669930 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.670541 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.670547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.670549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.670670 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.671000 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.671004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.671005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.671345 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:56.671580 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:56.671849 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:27:56.672061 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":394,"threads_started":1}
I20260806 22:27:56.673676 14797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:56.757772 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 14596
I20260806 22:27:56.780952 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 14812
Aug 06 22:27:56 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:27:56.797250 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:56.781922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.1.8.66:39243)
0806 22:27:56.782475 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:56.782482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:56.782507 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:27:56.782569 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:56.782575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:56.782778 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:56.782898 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:56.784450 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.785473 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.786773 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.787302 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.787396 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:56.791730 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:56.791741 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:56.791755 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:56.791795 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:56.794743 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.795352 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.795358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.795361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.795500 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.795804 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.795807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.795809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.796235 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:56.796485 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:56.796764 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:27:56.796961 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":286,"threads_started":1}
I20260806 22:27:56.798517 15000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:56.893649 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 14812
I20260806 22:27:56.919205 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15017
Aug 06 22:27:56 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:27:56.935322 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:56.920330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.1.8.67:35545)
0806 22:27:56.920941 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:56.920948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:56.920977 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:27:56.921039 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:56.921046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:56.921274 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:56.921392 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:56.922669 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.923655 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.925037 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:56.925589 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:56.925688 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:56.929971 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:56.929982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:56.929998 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:56.930032 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:56.932923 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.933509 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.933516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.933518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.933632 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:56.933920 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:56.933923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:56.933925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:56.934245 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:56.934512 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:56.934686 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:27:56.934910 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":326,"threads_started":1}
I20260806 22:27:56.936704 15226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:57.019860 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 15017
I20260806 22:27:57.042474 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 14453
I20260806 22:27:57.043682 14551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:27:57.148566 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 14453
I20260806 22:27:57.177572 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38487
--webserver_interface=127.1.8.126
--webserver_port=41317
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:27:57.525722 15264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:57.526072 15264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:57.526237 15264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:57.533447 15264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:27:57.533587 15264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:57.533691 15264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:57.533789 15264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:27:57.533878 15264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:27:57.543586 15264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=41317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:57.544970 15264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:57.546329 15264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:57.558497 15275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:57.558415 15276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:57.558465 15280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:57.559270 15264 server_base.cc:1034] running on GCE node
I20260806 22:27:57.560066 15264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:57.561556 15264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:57.562755 15264 hybrid_clock.cc:648] HybridClock initialized: now 1786055277562697 us; error 54 us; skew 500 ppm
I20260806 22:27:57.563117 15264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:57 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:57.566461 15264 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:27:57.567941 15264 webserver.cc:460] Webserver started at http://127.1.8.126:41317/ using document root <none> and password file <none>
I20260806 22:27:57.568694 15264 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:57.568816 15264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:57.574536 15264 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:27:57.577633 15291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:57.579082 15264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:57.579234 15264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "28a54f604c0040a7a7fcc25643340aba"
format_stamp: "Formatted at 2026-08-06 22:27:54 on dist-test-slave-7vjv"
server_key: "cfc4a973ec4f1048c167553d435b8b9d"
server_key_iv: "840a14907520733f1279bf1326ab606e"
server_key_version: "encryptionkey@0"
I20260806 22:27:57.580271 15264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:57.608741 15264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:57.612865 15264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:57.613108 15264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:57.632058 15264 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38487
I20260806 22:27:57.632082 15364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38487 every 8 connection(s)
I20260806 22:27:57.633649 15264 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:27:57.636479 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 15264
I20260806 22:27:57.637284 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:32853
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:38487
--webserver_port=45601
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:27:57.641067 15365 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:27:57.644424 15365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Bootstrap starting.
I20260806 22:27:57.669234 15365 log.cc:824] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Log is configured to *not* fsync() on all Append() calls
I20260806 22:27:57.680832 15365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:27:57.681540 15365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Bootstrap complete.
I20260806 22:27:57.695308 15365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:57.696516 15365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a54f604c0040a7a7fcc25643340aba, State: Initialized, Role: FOLLOWER
I20260806 22:27:57.697230 15365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:57.697511 15365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:27:57.697630 15365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:27:57.697813 15365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:27:57.703372 15365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:57.704012 15365 leader_election.cc:302] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28a54f604c0040a7a7fcc25643340aba; no voters:
I20260806 22:27:57.704674 15365 leader_election.cc:288] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:27:57.704821 15371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:27:57.706313 15371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [term 2 LEADER]: Becoming Leader. State: Replica: 28a54f604c0040a7a7fcc25643340aba, State: Running, Role: LEADER
I20260806 22:27:57.706980 15365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:27:57.706861 15371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } }
I20260806 22:27:57.709250 15379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28a54f604c0040a7a7fcc25643340aba. Latest consensus state: current_term: 2 leader_uuid: "28a54f604c0040a7a7fcc25643340aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } } }
I20260806 22:27:57.709679 15379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:57.710316 15375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "28a54f604c0040a7a7fcc25643340aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a54f604c0040a7a7fcc25643340aba" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38487 } } }
I20260806 22:27:57.710680 15375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba [sys.catalog]: This master's current role is: LEADER
I20260806 22:27:57.712973 15387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:27:57.719854 15387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:27:57.721346 15387 catalog_manager.cc:1259] Loaded cluster ID: 1f75f2c76d9843f8b0bc0fc6ed855307
I20260806 22:27:57.721498 15387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:27:57.725443 15387 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:27:57.726998 15387 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28a54f604c0040a7a7fcc25643340aba: Loaded TSK: 0
I20260806 22:27:57.728571 15387 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:27:58.006680 15367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:58.007020 15367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:58.007179 15367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:58.014667 15367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:58.014808 15367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:58.014976 15367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:27:58.025161 15367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.65.keytab
--principal=oryx/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:32853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=45601
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:58.026789 15367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:58.028242 15367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:58.043089 15418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.043313 15417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.043853 15420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:58.045173 15367 server_base.cc:1034] running on GCE node
I20260806 22:27:58.045872 15367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:58.046962 15367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:58.048230 15367 hybrid_clock.cc:648] HybridClock initialized: now 1786055278048169 us; error 54 us; skew 500 ppm
I20260806 22:27:58.048553 15367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055278, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:58.051831 15367 init.cc:377] Logged in from keytab as oryx/127.1.8.65@KRBTEST.COM (short username oryx)
I20260806 22:27:58.053444 15367 webserver.cc:460] Webserver started at http://127.1.8.65:45601/ using document root <none> and password file <none>
I20260806 22:27:58.054184 15367 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:58.054342 15367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:58.059984 15367 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:27:58.062862 15427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.064333 15367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:58.064505 15367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "f0d8f9cbe2f0467e9e3065b7a717da59"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "09ab4367ef11df777a04c7ef0576bb61"
server_key_iv: "8c1e4c6d35291be15cbd051faa45d12a"
server_key_version: "encryptionkey@0"
I20260806 22:27:58.065477 15367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:58.086172 15367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:58.090461 15367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:58.090764 15367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:58.091697 15367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:58.093112 15367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:58.093232 15367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.093365 15367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:58.093448 15367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.126296 15367 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:32853
I20260806 22:27:58.126401 15604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:32853 every 8 connection(s)
I20260806 22:27:58.128017 15367 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:27:58.134341 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 15367
I20260806 22:27:58.135147 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:39243
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:38487
--webserver_port=38205
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:58.148958 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.131162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:45245)
0806 22:27:58.132025 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.132084 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.133897 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.135654 (+ 1757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.135662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.136282 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.136847 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.136865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.139384 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.139389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.140752 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.140766 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.142463 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.142470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.142682 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.143681 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.143713 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.147659 (+ 3946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":400,"threads_started":1}
W20260806 22:27:58.149008 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.131068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:45245 (local address 127.1.8.126:38487)
0806 22:27:58.131794 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.131806 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.132234 (+ 428us) server_negotiation.cc:407] Connection header received
0806 22:27:58.134579 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.134598 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.135054 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.135491 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.137075 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.139191 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.140966 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.142183 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.142408 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.148183 (+ 5775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:45245: BlockingRecv error: recv got EOF from 127.1.8.65:45245 (error 108)
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":278,"threads_started":1}
W20260806 22:27:58.149817 15605 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:38487 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.8.126:38487: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:27:58.159657 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.150571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:51771 (local address 127.1.8.126:38487)
0806 22:27:58.150716 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.150719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.150738 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:27:58.150941 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.150945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.150999 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.151076 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.151956 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.152706 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.154594 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.154953 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.155061 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.159493 (+ 4432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:51771: BlockingRecv error: recv got EOF from 127.1.8.65:51771 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:27:58.159689 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.150481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:51771)
0806 22:27:58.150677 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.150698 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.150806 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.151130 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.151137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.151580 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.151786 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.151800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.152864 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.152871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.154466 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.154483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.155118 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.155126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.155340 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.155918 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.155950 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.159342 (+ 3392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:58.169808 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.160464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:48459)
0806 22:27:58.160637 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.160657 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.160752 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.161118 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.161124 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.161550 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.161755 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.161769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.162737 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.162745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.164462 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.164478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.165135 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.165144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.165361 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.165911 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.165939 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.169411 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:27:58.169853 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.160670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:48459 (local address 127.1.8.126:38487)
0806 22:27:58.160820 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.160823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.160836 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:27:58.160909 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.160912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.160971 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.161042 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.161931 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.162594 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.164599 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.164992 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.165083 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.169615 (+ 4532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:48459: BlockingRecv error: recv got EOF from 127.1.8.65:48459 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:27:58.170195 15605 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:27:58.486526 15610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:58.486814 15610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:58.486907 15610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:58.494220 15610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:58.494335 15610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:58.494448 15610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:27:58.504635 15610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.66.keytab
--principal=oryx/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:39243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=38205
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:58.505935 15610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:58.507184 15610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:58.520146 15617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.520118 15620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.520210 15618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:58.521245 15610 server_base.cc:1034] running on GCE node
I20260806 22:27:58.521771 15610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:58.522584 15610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:58.523784 15610 hybrid_clock.cc:648] HybridClock initialized: now 1786055278523747 us; error 30 us; skew 500 ppm
I20260806 22:27:58.524049 15610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055278, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:58.527314 15610 init.cc:377] Logged in from keytab as oryx/127.1.8.66@KRBTEST.COM (short username oryx)
I20260806 22:27:58.528877 15610 webserver.cc:460] Webserver started at http://127.1.8.66:38205/ using document root <none> and password file <none>
I20260806 22:27:58.529574 15610 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:58.529736 15610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:58.534973 15610 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 22:27:58.537539 15628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.538852 15610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:27:58.539021 15610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "76a3ef075d734ebbaefe10a584b470cd"
format_stamp: "Formatted at 2026-08-06 22:27:55 on dist-test-slave-7vjv"
server_key: "99c8195606276a57809a4862fd3c2073"
server_key_iv: "5a11bec58a01a2b58addb31b01e0bec5"
server_key_version: "encryptionkey@0"
I20260806 22:27:58.539986 15610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:58.560488 15610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:58.564600 15610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:58.565026 15610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:58.566011 15610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:58.567322 15610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:58.567451 15610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.567569 15610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:58.567654 15610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:58.601662 15610 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:39243
I20260806 22:27:58.601747 15796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:39243 every 8 connection(s)
I20260806 22:27:58.603106 15610 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:27:58.610065 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 15610
I20260806 22:27:58.610749 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:35545
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:38487
--webserver_port=39995
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:27:58.620599 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.605769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:35023 (local address 127.1.8.126:38487)
0806 22:27:58.605947 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.605952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.607129 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 22:27:58.608987 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.608992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.609059 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.609147 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.610639 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.611769 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.613295 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.613970 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.614104 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.620318 (+ 6214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:35023: BlockingRecv error: recv got EOF from 127.1.8.66:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:27:58.621405 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.606197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:35023)
0806 22:27:58.606969 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.607018 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.608737 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.609340 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.609347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.609870 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.610441 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.610459 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.611931 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.611934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.613157 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.613169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.614189 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.614197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.614473 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.615223 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.615258 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.620084 (+ 4826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":274,"threads_started":1}
W20260806 22:27:58.622046 15798 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:38487 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.8.126:38487: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:58.630391 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.622622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:54483)
0806 22:27:58.622810 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.622829 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.622935 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.623281 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.623286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.623609 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.623735 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.623743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.624714 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.624720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.625707 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.625718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.626416 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.626422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.626599 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.627172 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.627189 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.630083 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 22:27:58.630684 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.622809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:54483 (local address 127.1.8.126:38487)
0806 22:27:58.623029 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.623033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.623048 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:27:58.623088 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.623091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.623145 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.623253 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.623869 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.624585 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.625831 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.626264 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.626390 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.630310 (+ 3920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:54483: BlockingRecv error: recv got EOF from 127.1.8.66:54483 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:58.642403 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:58.631215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:47527)
0806 22:27:58.631381 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:58.631401 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:58.631511 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:58.632636 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:58.632640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:58.632990 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:58.633168 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.633179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.634783 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.634788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.636246 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:58.636261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.637782 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.637788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:58.637961 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.638614 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:58.638635 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:58.642121 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:27:58.642848 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:58.631466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:47527 (local address 127.1.8.126:38487)
0806 22:27:58.632316 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:58.632323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:58.632343 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:27:58.632394 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:58.632401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:58.632476 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:58.632582 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:58.633317 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.634626 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.636793 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:58.637618 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:58.637771 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:58.642389 (+ 4618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:47527: BlockingRecv error: recv got EOF from 127.1.8.66:47527 (error 108)
Metrics: {"server-negotiator.queue_time_us":681}
W20260806 22:27:58.642886 15798 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:27:58.949925 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:27:58.950230 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:27:58.950417 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:27:58.957698 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:27:58.957809 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:27:58.957934 15802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:27:58.967774 15802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/krb5kdc/oryx_127.1.8.67.keytab
--principal=oryx/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:35545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=39995
--tserver_master_addrs=127.1.8.126:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:27:58.969182 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:27:58.970546 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:27:58.983742 15811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.983867 15810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:27:58.984048 15813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:27:58.984979 15802 server_base.cc:1034] running on GCE node
I20260806 22:27:58.985605 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:27:58.986531 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:27:58.987756 15802 hybrid_clock.cc:648] HybridClock initialized: now 1786055278987724 us; error 64 us; skew 500 ppm
I20260806 22:27:58.988062 15802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:27:58 dist-test-slave-7vjv krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055278, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:27:58.991128 15802 init.cc:377] Logged in from keytab as oryx/127.1.8.67@KRBTEST.COM (short username oryx)
I20260806 22:27:58.992564 15802 webserver.cc:460] Webserver started at http://127.1.8.67:39995/ using document root <none> and password file <none>
I20260806 22:27:58.993242 15802 fs_manager.cc:359] Metadata directory not provided
I20260806 22:27:58.993391 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:27:58.998657 15802 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 22:27:59.001485 15822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:59.002815 15802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:27:59.002986 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "27f189ffa1524ce081c4ba7b59bb9eb4"
format_stamp: "Formatted at 2026-08-06 22:27:56 on dist-test-slave-7vjv"
server_key: "da7622e42a957adbb257a29a4096d805"
server_key_iv: "94fcdddf13febb907cd8ecb4d3d6f347"
server_key_version: "encryptionkey@0"
I20260806 22:27:59.003939 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:27:59.019364 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:27:59.023082 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:27:59.023340 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:27:59.024266 15802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:27:59.025465 15802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:27:59.025574 15802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:59.025707 15802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:27:59.025791 15802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:27:59.061581 15802 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:35545
I20260806 22:27:59.061636 15995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:35545 every 8 connection(s)
I20260806 22:27:59.063064 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:27:59.072819 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 15802
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:27:59.079929 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:59.065671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:59075 (local address 127.1.8.126:38487)
0806 22:27:59.065876 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.065881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.067047 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 22:27:59.069117 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.069121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.069193 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.069279 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.070905 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.071613 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.073351 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.073915 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.074051 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.079610 (+ 5559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:59075: BlockingRecv error: recv got EOF from 127.1.8.67:59075 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:27:59.080653 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.066049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:59075)
0806 22:27:59.066833 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:59.066902 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:59.068828 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:59.069495 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:59.069503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:59.070135 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:59.070737 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.070756 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.071746 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.071750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.073200 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.073214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.074163 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.074172 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.074390 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.075145 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:59.075179 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:59.079434 (+ 4255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":293,"threads_started":1}
W20260806 22:27:59.081409 15996 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:38487 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.8.126:38487: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:59.090634 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.082052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:50727)
0806 22:27:59.082233 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:59.082254 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:59.082391 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:59.082693 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:59.082700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:59.083166 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:59.083425 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.083442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.084504 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.084511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.085629 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.085642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.086455 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.086461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.086670 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.087177 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:59.087195 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:59.090339 (+ 3144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:27:59.090834 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:59.082112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:50727 (local address 127.1.8.126:38487)
0806 22:27:59.082252 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.082255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.082310 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:27:59.082500 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.082503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.082552 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.082624 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.083573 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.084367 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.085789 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.086288 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.086433 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.090526 (+ 4093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:50727: BlockingRecv error: recv got EOF from 127.1.8.67:50727 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:27:59.092149 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.073910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.1.8.126:38487)
0806 22:27:59.075602 (+ 1692us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.075606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.075619 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:27:59.075655 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.075658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.075704 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.075781 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.078042 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.078868 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.080463 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.081022 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.081173 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.084970 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:27:59.084977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:27:59.084990 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:27:59.085031 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:27:59.088362 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:59.089169 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:59.089180 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:59.089192 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:59.089414 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:27:59.089851 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:27:59.089856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:27:59.089860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:27:59.090503 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:27:59.090836 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 22:27:59.091350 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 22:27:59.091759 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1595,"thread_start_us":891,"threads_started":1}
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:59.099843 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.091550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:48255)
0806 22:27:59.091832 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:59.091862 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:59.091966 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:59.092502 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:59.092508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:59.092809 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:59.092963 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.092970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.094390 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.094396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.095647 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.095660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.096486 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.096491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.096635 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.097112 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:59.097129 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:59.099659 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143}
W20260806 22:27:59.100160 15996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:27:59.100184 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:59.091895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:48255 (local address 127.1.8.126:38487)
0806 22:27:59.092222 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.092227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.092244 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:27:59.092288 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.092295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.092371 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.092474 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.093094 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.094198 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.095779 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.096361 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.096457 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.099845 (+ 3388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:48255: BlockingRecv error: recv got EOF from 127.1.8.67:48255 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:59.179219 16003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.170726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:45343)
0806 22:27:59.171315 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:59.171331 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:59.171444 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:59.171717 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:59.171721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:59.172269 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:59.172488 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.172496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.173457 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.173461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.174456 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.174467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.175164 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.175168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.175298 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.175917 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:59.175934 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:59.178931 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":343,"threads_started":1}
W20260806 22:27:59.179469 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:59.170791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:45343 (local address 127.1.8.126:38487)
0806 22:27:59.170965 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.170969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.171402 (+ 433us) server_negotiation.cc:407] Connection header received
0806 22:27:59.171538 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.171542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.171601 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.171681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.172617 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.173342 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.174572 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.175005 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.175094 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.179198 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:45343: BlockingRecv error: recv got EOF from 127.1.8.65:45343 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:27:59 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:27:59.651883 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:27:59.643503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:34787)
0806 22:27:59.644068 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 22:27:59.644110 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:27:59.644256 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:27:59.644669 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:27:59.644674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:27:59.645294 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:27:59.645515 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.645525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.646492 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.646496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.647471 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:27:59.647482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.648181 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.648186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:27:59.648310 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.648859 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:27:59.648876 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:27:59.651651 (+ 2775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":278,"threads_started":1}
W20260806 22:27:59.652127 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:27:59.643575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:34787 (local address 127.1.8.126:38487)
0806 22:27:59.643798 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:27:59.643806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:27:59.644206 (+ 400us) server_negotiation.cc:407] Connection header received
0806 22:27:59.644390 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:27:59.644396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:27:59.644483 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:27:59.644600 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:27:59.645644 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.646372 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.647602 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:27:59.648004 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:27:59.648146 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:27:59.651849 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:34787: BlockingRecv error: recv got EOF from 127.1.8.66:34787 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:00.113497 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:00.100753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:55017)
0806 22:28:00.101347 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:00.101364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:00.101481 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:00.101787 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:00.101791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:00.102287 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:00.102526 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.102535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.103593 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.103599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.104843 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.104856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.105543 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.105549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.105699 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.110250 (+ 4551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:00.110268 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:00.113203 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":341,"threads_started":1}
W20260806 22:28:00.113674 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:00.100799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:55017 (local address 127.1.8.126:38487)
0806 22:28:00.100970 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:00.100974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:00.101431 (+ 457us) server_negotiation.cc:407] Connection header received
0806 22:28:00.101606 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:00.101610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:00.101670 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:00.101742 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:00.102655 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.103412 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.104961 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.105355 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.105445 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.113410 (+ 7965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:55017: BlockingRecv error: recv got EOF from 127.1.8.67:55017 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:00.188472 16010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:00.180219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:42635)
0806 22:28:00.180730 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:00.180753 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:00.180915 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:00.181295 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:00.181300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:00.181802 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:00.181948 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.181957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.182930 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.182934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.183900 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.183913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.184612 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.184616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.184729 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.185333 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:00.185350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:00.188235 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":274,"threads_started":1}
W20260806 22:28:00.188670 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:00.180249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:42635 (local address 127.1.8.126:38487)
0806 22:28:00.180448 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:00.180456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:00.180860 (+ 404us) server_negotiation.cc:407] Connection header received
0806 22:28:00.181044 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:00.181049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:00.181131 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:00.181230 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:00.182076 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.182813 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.184035 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.184458 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.184594 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.188398 (+ 3804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:42635: BlockingRecv error: recv got EOF from 127.1.8.65:42635 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:00 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:00.661242 16011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:00.652806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:53971)
0806 22:28:00.653322 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:00.653339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:00.653452 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:00.653783 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:00.653788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:00.654377 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:00.654536 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.654548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.655681 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.655687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.656693 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:00.656706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.657422 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.657428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:00.657610 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.658175 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:00.658193 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:00.660995 (+ 2802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
W20260806 22:28:00.661413 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:00.652837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:53971 (local address 127.1.8.126:38487)
0806 22:28:00.653001 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:00.653005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:00.653408 (+ 403us) server_negotiation.cc:407] Connection header received
0806 22:28:00.653559 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:00.653563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:00.653623 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:00.653730 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:00.654703 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.655527 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.656836 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:00.657261 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:00.657391 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:00.661192 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:53971: BlockingRecv error: recv got EOF from 127.1.8.66:53971 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:01.122638 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:01.114489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:38573)
0806 22:28:01.115097 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:01.115118 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:01.115248 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:01.115671 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:01.115677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:01.116140 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:01.116308 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.116316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.117276 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.117279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.118149 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.118159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.118801 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.118805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.118912 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.119564 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:01.119583 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:01.122381 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
W20260806 22:28:01.122836 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:01.115009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:38573 (local address 127.1.8.126:38487)
0806 22:28:01.115191 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:01.115195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:01.115212 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:28:01.115413 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:01.115418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:01.115501 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:01.115605 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:01.116446 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.117168 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.118260 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.118659 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.118746 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.122593 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:38573: BlockingRecv error: recv got EOF from 127.1.8.67:38573 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:01.197515 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:01.189397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:49651)
0806 22:28:01.189973 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:01.189989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:01.190101 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:01.190367 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:01.190370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:01.190735 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:01.190899 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.190906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.191869 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.191873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.192800 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.192812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.193453 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.193466 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.193588 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.194136 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:01.194152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:01.197204 (+ 3052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":306,"threads_started":1}
W20260806 22:28:01.197830 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:01.189439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:49651 (local address 127.1.8.126:38487)
0806 22:28:01.189590 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:01.189594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:01.190058 (+ 464us) server_negotiation.cc:407] Connection header received
0806 22:28:01.190187 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:01.190190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:01.190247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:01.190325 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:01.191033 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.191755 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.192926 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.193337 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.193429 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.197439 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:49651: BlockingRecv error: recv got EOF from 127.1.8.65:49651 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:01 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:01.670715 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:01.662163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:45059)
0806 22:28:01.662710 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:01.662727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:01.662854 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:01.663169 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:01.663173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:01.663528 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:01.663702 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.663710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.664867 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.664871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.665757 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:01.665767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.666419 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.666425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:01.666624 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.667298 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:01.667341 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:01.670359 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":287,"threads_started":1}
W20260806 22:28:01.670820 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:01.662244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:45059 (local address 127.1.8.126:38487)
0806 22:28:01.662412 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:01.662416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:01.662795 (+ 379us) server_negotiation.cc:407] Connection header received
0806 22:28:01.662966 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:01.662969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:01.663032 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:01.663105 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:01.663821 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.664741 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.665879 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:01.666267 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:01.666395 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:01.670572 (+ 4177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:45059: BlockingRecv error: recv got EOF from 127.1.8.66:45059 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:02.133095 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:02.123670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:35623)
0806 22:28:02.124258 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:02.124274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:02.124469 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:02.124826 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:02.124830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:02.125223 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:02.125437 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.125469 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.126666 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.126672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.127588 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.127600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.128762 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.128767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.129011 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.129800 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:02.129818 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:02.132831 (+ 3013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":285,"threads_started":1}
W20260806 22:28:02.133299 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:02.123740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:35623 (local address 127.1.8.126:38487)
0806 22:28:02.123985 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:02.123994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:02.124370 (+ 376us) server_negotiation.cc:407] Connection header received
0806 22:28:02.124620 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:02.124624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:02.124686 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:02.124793 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:02.125598 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.126503 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.127738 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.128423 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.128572 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.133077 (+ 4505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:35623: BlockingRecv error: recv got EOF from 127.1.8.67:35623 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:02.207118 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:02.198367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:44395)
0806 22:28:02.198862 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:02.198878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:02.199034 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:02.199288 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:02.199291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:02.199692 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:02.199902 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.199915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.201279 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.201282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.202276 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.202287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.202988 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.202992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.203128 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.203748 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:02.203766 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:02.206791 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":276,"threads_started":1}
W20260806 22:28:02.207456 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:02.198499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44395 (local address 127.1.8.126:38487)
0806 22:28:02.198752 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:02.198758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:02.198991 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:28:02.199127 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:02.199131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:02.199193 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:02.199276 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:02.200064 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.201172 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.202392 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.202870 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.202962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.207106 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:44395: BlockingRecv error: recv got EOF from 127.1.8.65:44395 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:02 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:02.680318 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:02.671641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:33989)
0806 22:28:02.672255 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:02.672271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:02.672430 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:02.672759 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:02.672764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:02.673191 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:02.673344 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.673353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.674541 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.674545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.675739 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:02.675752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.676473 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.676477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:02.676630 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.677155 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:02.677172 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:02.680012 (+ 2840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
W20260806 22:28:02.680438 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:02.671727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:33989 (local address 127.1.8.126:38487)
0806 22:28:02.671920 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:02.671927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:02.672352 (+ 425us) server_negotiation.cc:407] Connection header received
0806 22:28:02.672556 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:02.672563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:02.672629 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:02.672724 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:02.673488 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.674378 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.675859 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:02.676351 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:02.676445 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:02.680230 (+ 3785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:33989: BlockingRecv error: recv got EOF from 127.1.8.66:33989 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:03.143164 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:03.134127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:47649)
0806 22:28:03.134642 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:03.134658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:03.134774 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:03.135168 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:03.135173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:03.135715 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:03.135898 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.135908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.137311 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.137315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.138291 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.138302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.138994 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.139000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.139165 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.139925 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:03.139946 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:03.142876 (+ 2930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":301,"threads_started":1}
W20260806 22:28:03.143356 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:03.134200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:47649 (local address 127.1.8.126:38487)
0806 22:28:03.134479 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:03.134485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:03.134727 (+ 242us) server_negotiation.cc:407] Connection header received
0806 22:28:03.134893 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:03.134899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:03.134978 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:03.135107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:03.136065 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.137176 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.138407 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.138809 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.138899 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.143115 (+ 4216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:47649: BlockingRecv error: recv got EOF from 127.1.8.67:47649 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:03.216578 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:03.208287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:50947)
0806 22:28:03.209086 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:03.209102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:03.209215 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:03.209501 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:03.209505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:03.209927 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:03.210121 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.210129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.211115 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.211119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.212063 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.212075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.212755 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.212759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.212872 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.213413 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:03.213430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:03.216324 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":486,"threads_started":1}
W20260806 22:28:03.216744 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:03.208342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:50947 (local address 127.1.8.126:38487)
0806 22:28:03.208609 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:03.208618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:03.209183 (+ 565us) server_negotiation.cc:407] Connection header received
0806 22:28:03.209311 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:03.209316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:03.209379 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:03.209458 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:03.210245 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.210983 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.212208 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.212637 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.212730 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.216524 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:50947: BlockingRecv error: recv got EOF from 127.1.8.65:50947 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:03 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:03.690992 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:03.681324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:36425)
0806 22:28:03.681965 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:03.681991 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:03.682107 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:03.682391 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:03.682394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:03.682926 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:03.683117 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.683127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.684695 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.684699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.685661 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:03.685672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.686454 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.686458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:03.686600 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.687274 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:03.687292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:03.690659 (+ 3367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":376,"threads_started":1}
W20260806 22:28:03.691107 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:03.681485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:36425 (local address 127.1.8.126:38487)
0806 22:28:03.681685 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:03.681690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:03.682102 (+ 412us) server_negotiation.cc:407] Connection header received
0806 22:28:03.682214 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:03.682218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:03.682289 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:03.682382 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:03.683262 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.684557 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.685798 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:03.686289 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:03.686422 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:03.690863 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:36425: BlockingRecv error: recv got EOF from 127.1.8.66:36425 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:04.152956 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:04.144274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:34295)
0806 22:28:04.144992 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:04.145009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:04.145161 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:04.145500 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:04.145504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:04.145995 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:04.146156 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.146165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.147189 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.147194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.148217 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.148230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.148921 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.148927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.149106 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.149735 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:04.149752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:04.152580 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":403,"threads_started":1}
W20260806 22:28:04.153052 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:04.144311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:34295 (local address 127.1.8.126:38487)
0806 22:28:04.144497 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:04.144501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:04.145094 (+ 593us) server_negotiation.cc:407] Connection header received
0806 22:28:04.145291 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:04.145296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:04.145370 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:04.145469 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:04.146294 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.147046 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.148356 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.148771 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.148898 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.152791 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:34295: BlockingRecv error: recv got EOF from 127.1.8.67:34295 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:04.225829 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:04.217555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:60871)
0806 22:28:04.218114 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:04.218131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:04.218247 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:04.218526 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:04.218529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:04.218921 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:04.219092 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.219100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.220135 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.220139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.221062 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.221074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.221745 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.221749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.221867 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.222516 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:04.222536 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:04.225443 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":310,"threads_started":1}
W20260806 22:28:04.225941 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:04.217601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60871 (local address 127.1.8.126:38487)
0806 22:28:04.217784 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:04.217789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:04.218203 (+ 414us) server_negotiation.cc:407] Connection header received
0806 22:28:04.218338 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:04.218342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:04.218401 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:04.218481 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:04.219220 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.220002 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.221191 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.221606 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.221699 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.225666 (+ 3967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:60871: BlockingRecv error: recv got EOF from 127.1.8.65:60871 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:04 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:04.701085 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:04.692156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:49641)
0806 22:28:04.692674 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:04.692690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:04.692846 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:04.693172 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:04.693177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:04.693601 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:04.693755 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.693763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.694754 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.694760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.695707 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:04.695718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.696547 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.696553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:04.696725 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.697343 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:04.697362 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:04.700790 (+ 3428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":304,"threads_started":1}
W20260806 22:28:04.701238 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:04.692204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:49641 (local address 127.1.8.126:38487)
0806 22:28:04.692400 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:04.692409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:04.692780 (+ 371us) server_negotiation.cc:407] Connection header received
0806 22:28:04.692964 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:04.692970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:04.693040 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:04.693137 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:04.693913 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.694617 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.695857 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:04.696368 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:04.696515 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:04.701025 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:49641: BlockingRecv error: recv got EOF from 127.1.8.66:49641 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:05.166359 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:05.154029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:54405)
0806 22:28:05.154559 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:05.154577 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:05.154756 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:05.155164 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:05.155169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:05.155781 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:05.156088 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.156124 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.157230 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.157234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.158187 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.158199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.158918 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.158923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.159055 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.163311 (+ 4256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:05.163328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:05.166150 (+ 2822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
W20260806 22:28:05.166685 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:05.154105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:54405 (local address 127.1.8.126:38487)
0806 22:28:05.154338 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:05.154345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:05.154687 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:28:05.154903 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:05.154909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:05.154996 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:05.155086 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:05.156267 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.157081 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.158317 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.158749 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.158838 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.166409 (+ 7571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:54405: BlockingRecv error: recv got EOF from 127.1.8.67:54405 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:05.235285 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:05.226807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:60015)
0806 22:28:05.227438 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:05.227454 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:05.227632 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:05.227961 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:05.227966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:05.228490 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:05.228671 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.228680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.229713 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.229719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.230687 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.230698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.231434 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.231440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.231611 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.232293 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:05.232310 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:05.234915 (+ 2605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":313,"threads_started":1}
W20260806 22:28:05.235399 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:05.226925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60015 (local address 127.1.8.126:38487)
0806 22:28:05.227126 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:05.227134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:05.227558 (+ 424us) server_negotiation.cc:407] Connection header received
0806 22:28:05.227765 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:05.227775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:05.227832 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:05.227926 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:05.228834 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.229569 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.230830 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.231281 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.231405 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.235124 (+ 3719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:60015: BlockingRecv error: recv got EOF from 127.1.8.65:60015 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:05 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:05.710918 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:05.702053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:35935)
0806 22:28:05.702638 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:05.702655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:05.702775 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:05.703131 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:05.703135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:05.703650 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:05.703906 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.703918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.705275 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.705279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.706195 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:05.706205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.706881 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.706886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:05.707068 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.707795 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:05.707821 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:05.710649 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":255,"threads_started":1}
W20260806 22:28:05.711073 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:05.702174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:35935 (local address 127.1.8.126:38487)
0806 22:28:05.702499 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:05.702505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:05.702733 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:28:05.702873 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:05.702896 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:05.702983 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:05.703095 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:05.704048 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.705148 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.706315 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:05.706752 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:05.706843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:05.710822 (+ 3979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:35935: BlockingRecv error: recv got EOF from 127.1.8.66:35935 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:06.176525 16040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:06.167394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:48999)
0806 22:28:06.167993 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:06.168010 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:06.168215 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:06.168620 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:06.168626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:06.169075 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:06.169246 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.169254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.170482 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.170490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.171431 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.171443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.172209 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.172215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.172399 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.173065 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:06.173092 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:06.176091 (+ 2999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":266,"threads_started":1}
W20260806 22:28:06.176584 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:06.167463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:48999 (local address 127.1.8.126:38487)
0806 22:28:06.167702 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:06.167712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:06.168122 (+ 410us) server_negotiation.cc:407] Connection header received
0806 22:28:06.168372 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:06.168379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:06.168493 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:06.168587 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:06.169391 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.170341 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.171572 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.172024 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.172189 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.176353 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:48999: BlockingRecv error: recv got EOF from 127.1.8.67:48999 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:06.244830 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:06.236355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:36375)
0806 22:28:06.236900 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:06.236920 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:06.237073 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:06.237413 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:06.237419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:06.237872 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:06.238057 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.238066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.239096 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.239103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.240065 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.240077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.240890 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.240896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.241080 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.241638 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:06.241656 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:06.244607 (+ 2951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":298,"threads_started":1}
W20260806 22:28:06.245116 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:06.236579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:36375 (local address 127.1.8.126:38487)
0806 22:28:06.236824 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:06.236830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:06.237009 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:28:06.237202 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:06.237207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:06.237279 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:06.237375 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:06.238202 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.238939 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.240231 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.240696 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.240829 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.244849 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:36375: BlockingRecv error: recv got EOF from 127.1.8.65:36375 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:06 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:06.723664 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:06.711890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:59497)
0806 22:28:06.712371 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:06.712388 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:06.712503 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:06.712790 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:06.712793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:06.713227 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:06.713380 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.713388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.714553 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.714556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.715474 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:06.715485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.716251 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.716255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:06.716374 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.720615 (+ 4241us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:06.720633 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:06.723433 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":291,"threads_started":1}
W20260806 22:28:06.723831 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:06.712006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:59497 (local address 127.1.8.126:38487)
0806 22:28:06.712274 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:06.712281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:06.712472 (+ 191us) server_negotiation.cc:407] Connection header received
0806 22:28:06.712602 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:06.712605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:06.712666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:06.712743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:06.713505 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.714425 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.715592 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:06.716033 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:06.716204 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:06.723608 (+ 7404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:59497: BlockingRecv error: recv got EOF from 127.1.8.66:59497 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:07.185756 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:07.177526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:47023)
0806 22:28:07.178055 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:07.178070 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:07.178184 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:07.178469 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:07.178473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:07.178876 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:07.179073 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.179081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.180132 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.180139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.181055 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.181068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.181749 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.181754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.181870 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.182460 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:07.182479 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:07.185507 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":306,"threads_started":1}
W20260806 22:28:07.185917 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:07.177604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:47023 (local address 127.1.8.126:38487)
0806 22:28:07.177808 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:07.177813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:07.178144 (+ 331us) server_negotiation.cc:407] Connection header received
0806 22:28:07.178284 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:07.178287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:07.178344 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:07.178419 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:07.179207 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.179942 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.181181 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.181598 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.181692 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.185688 (+ 3996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:47023: BlockingRecv error: recv got EOF from 127.1.8.67:47023 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:07.255127 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:07.246009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:39189)
0806 22:28:07.246531 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:07.246548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:07.246670 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:07.247000 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:07.247004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:07.247404 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:07.247584 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.247592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.248714 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.248718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.249953 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.249970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.250814 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.250820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.251019 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.251625 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:07.251643 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:07.254761 (+ 3118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":277,"threads_started":1}
W20260806 22:28:07.255254 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:07.246098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:39189 (local address 127.1.8.126:38487)
0806 22:28:07.246368 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:07.246374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:07.246630 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:28:07.246772 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:07.246775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:07.246837 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:07.246955 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:07.247736 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.248595 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.250128 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.250657 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.250755 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.254995 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:39189: BlockingRecv error: recv got EOF from 127.1.8.65:39189 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:07 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:07.733470 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:07.724643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:60457)
0806 22:28:07.725188 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:07.725204 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:07.725381 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:07.725690 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:07.725693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:07.726090 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:07.726264 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.726272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.727325 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.727331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.728621 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:07.728633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.729453 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.729460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:07.729591 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.730167 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:07.730185 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:07.733078 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
W20260806 22:28:07.733536 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:07.724708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:60457 (local address 127.1.8.126:38487)
0806 22:28:07.724865 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:07.724869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:07.725274 (+ 405us) server_negotiation.cc:407] Connection header received
0806 22:28:07.725497 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:07.725500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:07.725567 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:07.725644 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:07.726380 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.727184 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.728769 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:07.729301 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:07.729397 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:07.733301 (+ 3904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:60457: BlockingRecv error: recv got EOF from 127.1.8.66:60457 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:08.195785 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:08.186867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:48673)
0806 22:28:08.187404 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:08.187420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:08.187586 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:08.187951 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:08.187975 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:08.188586 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:08.188879 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.188896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.190017 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.190026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.191110 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.191124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.191797 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.191801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.191914 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.192538 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:08.192555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:08.195435 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":278,"threads_started":1}
W20260806 22:28:08.195948 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:08.186866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:48673 (local address 127.1.8.126:38487)
0806 22:28:08.187090 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:08.187098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:08.187519 (+ 421us) server_negotiation.cc:407] Connection header received
0806 22:28:08.187736 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:08.187740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:08.187803 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:08.187933 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:08.189113 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.189872 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.191252 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.191671 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.191764 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.195682 (+ 3918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:48673: BlockingRecv error: recv got EOF from 127.1.8.67:48673 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:08.265368 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:08.256278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:44435)
0806 22:28:08.257067 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:08.257083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:08.257194 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:08.257549 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:08.257554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:08.257974 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:08.258117 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.258126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.259209 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.259215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.260205 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.260217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.260992 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.260998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.261176 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.261807 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:08.261824 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:08.264998 (+ 3174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":553,"threads_started":1}
W20260806 22:28:08.265499 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:08.256351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44435 (local address 127.1.8.126:38487)
0806 22:28:08.256580 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:08.256588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:08.257175 (+ 587us) server_negotiation.cc:407] Connection header received
0806 22:28:08.257330 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:08.257340 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:08.257403 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:08.257510 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:08.258272 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.259060 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.260351 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.260823 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.260975 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.265200 (+ 4225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:44435: BlockingRecv error: recv got EOF from 127.1.8.65:44435 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:08 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:08.742628 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:08.734522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:52285)
0806 22:28:08.735117 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:08.735138 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:08.735263 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:08.735642 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:08.735647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:08.736190 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:08.736359 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.736368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.737339 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.737342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.738253 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:08.738263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.738894 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.738898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:08.739018 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.739579 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:08.739608 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:08.742399 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":465,"threads_started":1}
W20260806 22:28:08.742858 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:08.734582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:52285 (local address 127.1.8.126:38487)
0806 22:28:08.734764 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:08.734769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:08.735219 (+ 450us) server_negotiation.cc:407] Connection header received
0806 22:28:08.735394 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:08.735399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:08.735483 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:08.735572 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:08.736501 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.737230 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.738373 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:08.738768 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:08.738859 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:08.742580 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:52285: BlockingRecv error: recv got EOF from 127.1.8.66:52285 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:09.205699 16051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:09.196957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:39265)
0806 22:28:09.197547 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:09.197562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:09.197714 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:09.198056 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:09.198061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:09.198513 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:09.198659 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.198668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.199860 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.199865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.200889 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.200901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.201645 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.201651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.201834 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.202460 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:09.202478 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:09.205424 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":285,"threads_started":1}
W20260806 22:28:09.205957 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:09.197179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:39265 (local address 127.1.8.126:38487)
0806 22:28:09.197471 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:09.197477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:09.197640 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:28:09.197839 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:09.197845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:09.197917 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:09.198018 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:09.198808 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.199706 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.201042 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.201484 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.201630 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.205696 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:39265: BlockingRecv error: recv got EOF from 127.1.8.67:39265 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:09.275161 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:09.266321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:57041)
0806 22:28:09.266788 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:09.266804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:09.266916 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:09.267171 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:09.267173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:09.267696 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:09.267871 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.267880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.269140 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.269144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.270158 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.270170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.270891 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.270894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.271050 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.271660 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:09.271678 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:09.274787 (+ 3109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":264,"threads_started":1}
W20260806 22:28:09.275192 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:09.266417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:57041 (local address 127.1.8.126:38487)
0806 22:28:09.266692 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:09.266698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:09.266882 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:28:09.267010 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:09.267014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:09.267072 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:09.267150 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:09.268027 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.269003 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.270299 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.270752 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.270849 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.274997 (+ 4148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:57041: BlockingRecv error: recv got EOF from 127.1.8.65:57041 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:09 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:09.752285 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:09.743460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:41347)
0806 22:28:09.744031 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:09.744051 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:09.744230 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:09.744586 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:09.744594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:09.745061 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:09.745257 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.745267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.746335 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.746341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.747261 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:09.747272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.748031 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.748037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:09.748245 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.748808 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:09.748826 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:09.751983 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
W20260806 22:28:09.752461 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:09.743572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:41347 (local address 127.1.8.126:38487)
0806 22:28:09.743787 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:09.743796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:09.744148 (+ 352us) server_negotiation.cc:407] Connection header received
0806 22:28:09.744359 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:09.744365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:09.744440 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:09.744543 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:09.745406 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.746190 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.747403 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:09.747862 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:09.748015 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:09.752247 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:41347: BlockingRecv error: recv got EOF from 127.1.8.66:41347 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:10.219791 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:10.206670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:33141)
0806 22:28:10.207231 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:10.207251 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:10.207370 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:10.207740 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:10.207745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:10.208242 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:10.208427 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.208436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.209430 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.209437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.210360 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.210371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.211051 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.211057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.211244 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.216024 (+ 4780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:10.216043 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:10.219415 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":325,"threads_started":1}
W20260806 22:28:10.219873 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:10.206772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:33141 (local address 127.1.8.126:38487)
0806 22:28:10.206949 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:10.206954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:10.207328 (+ 374us) server_negotiation.cc:407] Connection header received
0806 22:28:10.207480 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:10.207484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:10.207596 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:10.207703 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:10.208576 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.209299 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.210509 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.210894 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.211033 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.219629 (+ 8596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:33141: BlockingRecv error: recv got EOF from 127.1.8.67:33141 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:10.285766 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:10.276476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:42507)
0806 22:28:10.277591 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:10.277613 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:10.277821 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:10.278214 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:10.278219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:10.278719 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:10.278886 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.278895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.279987 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.279995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.280934 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.280946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.281885 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.281891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.282079 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.282643 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:10.282660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:10.285542 (+ 2882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":761,"threads_started":1}
W20260806 22:28:10.286127 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:10.276496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:42507 (local address 127.1.8.126:38487)
0806 22:28:10.276912 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:10.276916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:10.277728 (+ 812us) server_negotiation.cc:407] Connection header received
0806 22:28:10.277994 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:10.277999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:10.278070 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:10.278176 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:10.279052 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.279828 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.281085 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.281732 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.281867 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.285742 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:42507: BlockingRecv error: recv got EOF from 127.1.8.65:42507 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:10 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:10.762207 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:10.753322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:48287)
0806 22:28:10.753787 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:10.753805 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:10.753962 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:10.754256 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:10.754259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:10.754652 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:10.754809 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.754817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.755969 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.755973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.757415 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:10.757428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.758089 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.758094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:10.758220 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.758799 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:10.758816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:10.761863 (+ 3047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":270,"threads_started":1}
W20260806 22:28:10.762320 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:10.753420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:48287 (local address 127.1.8.126:38487)
0806 22:28:10.753686 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:10.753693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:10.753893 (+ 200us) server_negotiation.cc:407] Connection header received
0806 22:28:10.754068 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:10.754071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:10.754134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:10.754213 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:10.754935 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.755803 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.757545 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:10.757966 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:10.758063 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:10.762089 (+ 4026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:48287: BlockingRecv error: recv got EOF from 127.1.8.66:48287 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:11.229559 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:11.220970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:50907)
0806 22:28:11.221559 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:11.221575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:11.221744 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:11.222126 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:11.222131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:11.222568 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:11.222722 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.222730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.223788 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.223794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.224760 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.224783 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.225527 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.225533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.225724 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.226286 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:11.226302 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:11.229206 (+ 2904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":370,"threads_started":1}
W20260806 22:28:11.229686 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:11.220983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:50907 (local address 127.1.8.126:38487)
0806 22:28:11.221156 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:11.221164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:11.221667 (+ 503us) server_negotiation.cc:407] Connection header received
0806 22:28:11.221891 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:11.221897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:11.221975 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:11.222080 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:11.222869 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.223607 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.224911 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.225365 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.225499 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.229441 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:50907: BlockingRecv error: recv got EOF from 127.1.8.67:50907 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:11.295513 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:11.286710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:59021)
0806 22:28:11.287241 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:11.287257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:11.287419 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:11.287785 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:11.287790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:11.288256 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:11.288425 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.288433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.289486 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.289498 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.290456 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.290467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.291388 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.291395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.291553 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.292292 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:11.292322 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:11.295250 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":274,"threads_started":1}
W20260806 22:28:11.295693 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:11.286759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:59021 (local address 127.1.8.126:38487)
0806 22:28:11.286976 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:11.287003 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:11.287348 (+ 345us) server_negotiation.cc:407] Connection header received
0806 22:28:11.287546 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:11.287557 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:11.287641 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:11.287748 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:11.288584 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.289334 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.290606 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.291180 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.291310 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.295425 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:59021: BlockingRecv error: recv got EOF from 127.1.8.65:59021 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:11 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:11.772504 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:11.763325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:33123)
0806 22:28:11.763950 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:11.763974 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:11.764160 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:11.764501 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:11.764507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:11.764971 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:11.765140 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.765150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.766237 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.766243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.767495 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:11.767511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.768440 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.768448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:11.768635 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.769272 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:11.769290 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:11.772150 (+ 2860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":334,"threads_started":1}
W20260806 22:28:11.772627 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:11.763506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:33123 (local address 127.1.8.126:38487)
0806 22:28:11.763730 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:11.763738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:11.764067 (+ 329us) server_negotiation.cc:407] Connection header received
0806 22:28:11.764294 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:11.764299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:11.764360 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:11.764476 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:11.765288 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.766093 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.767649 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:11.768228 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:11.768359 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:11.772373 (+ 4014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:33123: BlockingRecv error: recv got EOF from 127.1.8.66:33123 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:12.241050 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:12.230729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:36789)
0806 22:28:12.231304 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:12.231321 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:12.231486 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:12.231843 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:12.231848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:12.232398 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:12.232575 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.232584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.233807 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.233812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.235168 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.235187 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.236401 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.236406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.236523 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.237297 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:12.237328 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:12.240807 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":277,"threads_started":1}
W20260806 22:28:12.241272 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:12.230842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:36789 (local address 127.1.8.126:38487)
0806 22:28:12.231087 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:12.231094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:12.231412 (+ 318us) server_negotiation.cc:407] Connection header received
0806 22:28:12.231628 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:12.231633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:12.231707 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:12.231812 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:12.232726 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.233467 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.235317 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.236208 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.236377 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.241038 (+ 4661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:36789: BlockingRecv error: recv got EOF from 127.1.8.67:36789 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:12.304708 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:12.296512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:51973)
0806 22:28:12.297047 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:12.297065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:12.297179 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:12.297450 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:12.297453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:12.297826 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:12.298002 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.298010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.298986 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.298989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.299938 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.299948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.300667 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.300672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.300814 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.301428 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:12.301447 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:12.304407 (+ 2960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":296,"threads_started":1}
W20260806 22:28:12.305166 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:12.296586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:51973 (local address 127.1.8.126:38487)
0806 22:28:12.296868 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:12.296872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:12.297140 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:28:12.297272 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:12.297275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:12.297333 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:12.297403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:12.298122 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.298867 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.300067 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.300504 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.300598 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.304708 (+ 4110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:51973: BlockingRecv error: recv got EOF from 127.1.8.65:51973 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:12 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:12.782279 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:12.773644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:40851)
0806 22:28:12.774265 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:12.774284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:12.774413 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:12.774766 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:12.774769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:12.775201 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:12.775399 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.775408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.776584 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.776589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.777545 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:12.777563 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.778302 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.778306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:12.778452 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.779120 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:12.779139 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:12.782013 (+ 2874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":357,"threads_started":1}
W20260806 22:28:12.782416 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:12.773686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:40851 (local address 127.1.8.126:38487)
0806 22:28:12.773854 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:12.773858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:12.774362 (+ 504us) server_negotiation.cc:407] Connection header received
0806 22:28:12.774555 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:12.774559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:12.774625 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:12.774713 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:12.775534 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.776417 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.777700 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:12.778148 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:12.778245 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:12.782210 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:40851: BlockingRecv error: recv got EOF from 127.1.8.66:40851 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.67@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:13.250820 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:13.242115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.67:37149)
0806 22:28:13.242581 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:13.242598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:13.242716 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:13.243044 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:13.243048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:13.243471 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:13.243659 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.243668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.244871 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.244875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.245813 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.245825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.246482 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.246486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.246606 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.247209 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:13.247226 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:13.250453 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
W20260806 22:28:13.251001 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:13.242158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:37149 (local address 127.1.8.126:38487)
0806 22:28:13.242402 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:13.242409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:13.242696 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:28:13.242818 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:13.242821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:13.242882 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:13.243023 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:13.243790 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.244712 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.245935 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.246351 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.246448 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.250728 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.67:37149: BlockingRecv error: recv got EOF from 127.1.8.67:37149 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.65@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:13.315013 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:13.305904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.65:44229)
0806 22:28:13.306436 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:13.306452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:13.306592 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:13.306921 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:13.306924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:13.307525 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:13.307722 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.307731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.308934 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.308938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.310186 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.310198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.311075 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.311080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.311254 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.311935 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:13.311952 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:13.314728 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":310,"threads_started":1}
W20260806 22:28:13.315177 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:13.305956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44229 (local address 127.1.8.126:38487)
0806 22:28:13.306162 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:13.306166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:13.306545 (+ 379us) server_negotiation.cc:407] Connection header received
0806 22:28:13.306715 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:13.306719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:13.306795 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:13.306878 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:13.307882 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.308795 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.310327 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.310926 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.311060 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.314914 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.65:44229: BlockingRecv error: recv got EOF from 127.1.8.65:44229 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:28:13 dist-test-slave-7vjv krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.8.66@KRBTEST.COM for oryx/127.1.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:28:13.792382 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:13.783640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38487 (local address 127.1.8.66:46983)
0806 22:28:13.784250 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:13.784267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:13.784411 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:13.784803 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:13.784806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:13.785272 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:13.785471 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.785481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.786497 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.786500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.787430 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:13.787440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.788239 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.788243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:13.788458 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.789158 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:13.789176 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:13.792054 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.8.126:38487: Server oryx/127.1.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":328,"threads_started":1}
W20260806 22:28:13.792579 15608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:13.783671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:46983 (local address 127.1.8.126:38487)
0806 22:28:13.783846 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:13.783851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:13.784360 (+ 509us) server_negotiation.cc:407] Connection header received
0806 22:28:13.784525 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:13.784532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:13.784645 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:13.784767 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:13.785608 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.786360 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.787565 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:13.788026 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:13.788214 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:13.792311 (+ 4097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.8.66:46983: BlockingRecv error: recv got EOF from 127.1.8.66:46983 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:28:14.083285 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15367
I20260806 22:28:14.095507 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:14.084525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38768 (local address 127.1.8.65:32853)
0806 22:28:14.085088 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:14.085096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:14.085131 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:28:14.085196 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:14.085203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:14.085413 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:14.085585 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:14.086967 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.088004 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.089390 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.089944 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.090049 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:14.092595 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:14.092604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:14.092618 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:14.092652 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:14.094965 (+ 2313us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:28:14.095239 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38768: Request ticket server kudu/127.1.8.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":262,"threads_started":1}
W20260806 22:28:14.095628 16078 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38768: Request ticket server kudu/127.1.8.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:28:14.095723 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15367: Not authorized: Client connection negotiation failed: client connection to 127.1.8.65:32853: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:28:14.095940 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 15367
I20260806 22:28:14.119360 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15610
I20260806 22:28:14.131398 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:14.120488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.1.8.66:39243)
0806 22:28:14.121070 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:14.121078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:14.121107 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:28:14.121164 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:14.121170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:14.121402 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:14.121545 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:14.122778 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.123808 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.125038 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.125664 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.125755 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:14.128273 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:14.128281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:14.128294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:14.128331 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:14.130906 (+ 2575us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:28:14.131152 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41852: Request ticket server kudu/127.1.8.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":284,"threads_started":1}
W20260806 22:28:14.131524 16086 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41852: Request ticket server kudu/127.1.8.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:28:14.131522 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15610: Not authorized: Client connection negotiation failed: client connection to 127.1.8.66:39243: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:28:14.131685 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 15610
I20260806 22:28:14.160833 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15802
I20260806 22:28:14.172799 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:14.161747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.1.8.67:35545)
0806 22:28:14.162395 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:14.162402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:14.162429 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:28:14.162499 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:14.162506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:14.162727 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:14.162899 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:14.164232 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.165274 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.166483 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.167001 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.167122 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:14.169702 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:14.169711 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:14.169723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:14.169755 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:14.172329 (+ 2574us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:28:14.172572 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41144: Request ticket server kudu/127.1.8.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":330,"threads_started":1}
W20260806 22:28:14.172951 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15802: Not authorized: Client connection negotiation failed: client connection to 127.1.8.67:35545: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.8.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:28:14.173043 16088 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41144: Request ticket server kudu/127.1.8.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:28:14.173113 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 15802
I20260806 22:28:14.197165 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 15264
I20260806 22:28:14.198482 15359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:14.319392 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 15264
2026-08-06T22:28:14Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21866 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:28:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:28:14Z Disabled control of system clock
I20260806 22:28:14.424314 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34163
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:28:14.764845 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:14.765126 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:14.765204 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:14.772339 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:28:14.772436 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:14.772485 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:14.772542 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:28:14.772583 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:28:14.781253 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34163
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:14.782420 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:14.784221 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:14.796411 16105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:14.796311 16102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:14.797013 16094 server_base.cc:1034] running on GCE node
W20260806 22:28:14.797335 16101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:14.798271 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:14.799723 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:14.800964 16094 hybrid_clock.cc:648] HybridClock initialized: now 1786055294800951 us; error 67 us; skew 500 ppm
I20260806 22:28:14.801333 16094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:14.803690 16094 webserver.cc:460] Webserver started at http://127.1.8.126:41705/ using document root <none> and password file <none>
I20260806 22:28:14.804493 16094 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:14.804632 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:14.804977 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:14.807904 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "b05aaa239ddd40f8acbb74fdf556eb38"
format_stamp: "Formatted at 2026-08-06 22:28:14 on dist-test-slave-7vjv"
server_key: "7f3db55bdf452fd5c760c36bfd033f88"
server_key_iv: "e85cebd76dc804cb29e83415729c8ef1"
server_key_version: "encryptionkey@0"
I20260806 22:28:14.808777 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "b05aaa239ddd40f8acbb74fdf556eb38"
format_stamp: "Formatted at 2026-08-06 22:28:14 on dist-test-slave-7vjv"
server_key: "7f3db55bdf452fd5c760c36bfd033f88"
server_key_iv: "e85cebd76dc804cb29e83415729c8ef1"
server_key_version: "encryptionkey@0"
I20260806 22:28:14.814110 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:28:14.817632 16112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:14.819003 16094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:28:14.819199 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "b05aaa239ddd40f8acbb74fdf556eb38"
format_stamp: "Formatted at 2026-08-06 22:28:14 on dist-test-slave-7vjv"
server_key: "7f3db55bdf452fd5c760c36bfd033f88"
server_key_iv: "e85cebd76dc804cb29e83415729c8ef1"
server_key_version: "encryptionkey@0"
I20260806 22:28:14.819429 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:14.836587 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:14.837517 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:14.837812 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:14.856875 16094 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34163
I20260806 22:28:14.856900 16194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34163 every 8 connection(s)
I20260806 22:28:14.858316 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:28:14.862043 16196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:14.868032 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 16094
I20260806 22:28:14.868321 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:28:14.869268 1057 external_mini_cluster.cc:1428] Setting key 55179f71f56f05ffed4ae941d72915a2
I20260806 22:28:14.873942 16196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38: Bootstrap starting.
I20260806 22:28:14.877465 16196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:14.878732 16196 log.cc:824] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38: Log is configured to *not* fsync() on all Append() calls
I20260806 22:28:14.882290 16196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38: No bootstrap required, opened a new log
I20260806 22:28:14.884573 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:14.871642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.1.8.126:34163)
0806 22:28:14.872792 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:14.872804 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:14.872852 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:28:14.874797 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:14.874821 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:14.875312 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:14.875680 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:14.876387 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.877732 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.879586 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:14.880645 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:14.880786 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:14.882087 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:14.882095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:14.882108 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:14.882749 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:14.882831 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:14.882852 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:14.882954 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:14.883195 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 22:28:14.883539 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":464,"threads_started":1}
I20260806 22:28:14.890081 16196 raft_consensus.cc:348] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } }
I20260806 22:28:14.890461 16196 raft_consensus.cc:374] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:14.890576 16196 raft_consensus.cc:729] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05aaa239ddd40f8acbb74fdf556eb38, State: Initialized, Role: FOLLOWER
I20260806 22:28:14.891155 16196 consensus_queue.cc:260] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } }
I20260806 22:28:14.891408 16196 raft_consensus.cc:388] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:28:14.891573 16196 raft_consensus.cc:482] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:28:14.891776 16196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:14.893983 16196 raft_consensus.cc:504] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } }
I20260806 22:28:14.894465 16196 leader_election.cc:302] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b05aaa239ddd40f8acbb74fdf556eb38; no voters:
I20260806 22:28:14.895093 16196 leader_election.cc:288] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:28:14.895265 16205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:28:14.896008 16205 raft_consensus.cc:686] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [term 1 LEADER]: Becoming Leader. State: Replica: b05aaa239ddd40f8acbb74fdf556eb38, State: Running, Role: LEADER
I20260806 22:28:14.896674 16205 consensus_queue.cc:237] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } }
I20260806 22:28:14.897265 16196 sys_catalog.cc:564] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:28:14.899418 16207 sys_catalog.cc:455] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b05aaa239ddd40f8acbb74fdf556eb38. Latest consensus state: current_term: 1 leader_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } } }
I20260806 22:28:14.899768 16207 sys_catalog.cc:458] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:14.901549 16212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:28:14.900522 16206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05aaa239ddd40f8acbb74fdf556eb38" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34163 } } }
I20260806 22:28:14.903360 16206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:14.907238 16212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:28:14.916254 16212 catalog_manager.cc:1347] Generated new cluster ID: cd8615954b5b4ca19b3df6f3ca2ff0f9
I20260806 22:28:14.916405 16212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:28:14.935392 16212 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:28:14.936981 16212 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:28:14.945398 16212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b05aaa239ddd40f8acbb74fdf556eb38: Generated new TSK 0
I20260806 22:28:14.946288 16212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:28:14.952695 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--builtin_ntp_servers=127.1.8.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:28:15.315770 16233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:15.316071 16233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:15.316282 16233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:15.324023 16233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:15.324209 16233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:15.324433 16233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:28:15.334450 16233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:15.335983 16233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:15.338035 16233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:15.352432 16246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:15.352458 16242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:15.353121 16240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:15.353137 16233 server_base.cc:1034] running on GCE node
I20260806 22:28:15.353960 16233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:15.354889 16233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:15.356146 16233 hybrid_clock.cc:648] HybridClock initialized: now 1786055295356054 us; error 37 us; skew 500 ppm
I20260806 22:28:15.356504 16233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:15.361649 16233 webserver.cc:460] Webserver started at http://127.1.8.65:40047/ using document root <none> and password file <none>
I20260806 22:28:15.362459 16233 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:15.362617 16233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:15.363351 16233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:15.366766 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "e7f6e71064ca45f29f14a11fba6198f8"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "e5d9be9c401c86be7910e1a876612d48"
server_key_iv: "fb93440c2189073a8726ec2d272490fd"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.367719 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "e7f6e71064ca45f29f14a11fba6198f8"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "e5d9be9c401c86be7910e1a876612d48"
server_key_iv: "fb93440c2189073a8726ec2d272490fd"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.373723 16233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:28:15.377326 16256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.378577 16233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:28:15.378811 16233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "e7f6e71064ca45f29f14a11fba6198f8"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "e5d9be9c401c86be7910e1a876612d48"
server_key_iv: "fb93440c2189073a8726ec2d272490fd"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.379040 16233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:15.403002 16233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:15.404035 16233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:15.404415 16233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:15.405469 16233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:15.407248 16233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:15.407377 16233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.407528 16233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:15.407625 16233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.446269 16233 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:37335
I20260806 22:28:15.446334 16430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:37335 every 8 connection(s)
I20260806 22:28:15.447810 16233 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:28:15.455207 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 16233
I20260806 22:28:15.455447 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:28:15.456213 1057 external_mini_cluster.cc:1428] Setting key cff394b66a36ac94533acb825c4b0762
I20260806 22:28:15.459438 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--builtin_ntp_servers=127.1.8.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:28:15.460932 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:15.450444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:53117 (local address 127.1.8.126:34163)
0806 22:28:15.450917 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:15.450924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:15.452221 (+ 1297us) server_negotiation.cc:407] Connection header received
0806 22:28:15.453976 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:15.453981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:15.454063 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:15.454175 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:15.455211 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.456034 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.457814 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.458377 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.458488 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:15.460077 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:15.460083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:15.460086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:15.460165 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:15.460207 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:15.460220 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:15.460305 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:15.460605 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:28:15.460692 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260806 22:28:15.462417 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:15.451129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34163 (local address 127.1.8.65:53117)
0806 22:28:15.452023 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:15.452066 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:15.453711 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:15.454353 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:15.454360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:15.454442 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:15.455039 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:15.455056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.456234 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.456239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:15.457647 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:15.457675 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.458623 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.458631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:15.458908 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:15.459688 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:15.459694 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:15.459748 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:15.459848 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:15.460346 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:15.460368 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:15.460380 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:28:15.460590 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 22:28:15.461011 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":281,"threads_started":1}
I20260806 22:28:15.463907 16431 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34163
I20260806 22:28:15.464282 16431 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:15.465183 16431 heartbeater.cc:511] Master 127.1.8.126:34163 requested a full tablet report, sending...
I20260806 22:28:15.467473 16137 ts_manager.cc:194] Registered new tserver with Master: e7f6e71064ca45f29f14a11fba6198f8 (127.1.8.65:37335)
I20260806 22:28:15.469156 16137 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.65:53117
W20260806 22:28:15.802382 16435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:15.802683 16435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:15.802819 16435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:15.810463 16435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:15.810564 16435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:15.810722 16435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:28:15.820055 16435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:15.821429 16435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:15.823243 16435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:15.836107 16441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:15.836197 16445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:15.836162 16442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:15.837152 16435 server_base.cc:1034] running on GCE node
I20260806 22:28:15.837729 16435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:15.838641 16435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:15.839902 16435 hybrid_clock.cc:648] HybridClock initialized: now 1786055295839850 us; error 67 us; skew 500 ppm
I20260806 22:28:15.840303 16435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:15.842499 16435 webserver.cc:460] Webserver started at http://127.1.8.66:42419/ using document root <none> and password file <none>
I20260806 22:28:15.843168 16435 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:15.843302 16435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:15.843637 16435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:15.846395 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "2dfa269051ef4a818575b88b6e0d797b"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "1c2e0beb133daebfcc7306466d4ad6af"
server_key_iv: "7f4c483d776e7e2401c81467a2cd8e54"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.847157 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "2dfa269051ef4a818575b88b6e0d797b"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "1c2e0beb133daebfcc7306466d4ad6af"
server_key_iv: "7f4c483d776e7e2401c81467a2cd8e54"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.852059 16435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:28:15.855348 16456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.856679 16435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:28:15.856853 16435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "2dfa269051ef4a818575b88b6e0d797b"
format_stamp: "Formatted at 2026-08-06 22:28:15 on dist-test-slave-7vjv"
server_key: "1c2e0beb133daebfcc7306466d4ad6af"
server_key_iv: "7f4c483d776e7e2401c81467a2cd8e54"
server_key_version: "encryptionkey@0"
I20260806 22:28:15.857086 16435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:15.883288 16435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:15.884395 16435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:15.884690 16435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:15.885618 16435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:15.887094 16435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:15.887228 16435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.887377 16435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:15.887475 16435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:15.919906 16435 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:43471
I20260806 22:28:15.920004 16635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:43471 every 8 connection(s)
I20260806 22:28:15.921273 16435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:28:15.931174 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 16435
I20260806 22:28:15.931418 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:28:15.931897 1057 external_mini_cluster.cc:1428] Setting key 360421c139178495e6592c6c4760fc85
I20260806 22:28:15.934224 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:15.923845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:43095 (local address 127.1.8.126:34163)
0806 22:28:15.924040 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:15.924044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:15.925480 (+ 1436us) server_negotiation.cc:407] Connection header received
0806 22:28:15.927115 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:15.927119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:15.927183 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:15.927269 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:15.928284 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.929027 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.931318 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.931806 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.931910 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:15.933414 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:15.933421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:15.933426 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:15.933446 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:15.933494 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:15.933509 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:15.933602 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:15.933857 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 22:28:15.933953 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:28:15.935140 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--builtin_ntp_servers=127.1.8.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:28:15.935489 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:15.924446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34163 (local address 127.1.8.66:43095)
0806 22:28:15.925322 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:15.925357 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:15.926890 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:15.927465 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:15.927472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:15.927554 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:15.928079 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:15.928149 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.929294 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.929300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:15.931121 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:15.931138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:15.932056 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:15.932065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:15.932369 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:15.933125 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:15.933128 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:15.933164 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:15.933236 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:15.933605 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:15.933612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:15.933618 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:28:15.933800 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 22:28:15.934146 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":276,"threads_started":1}
I20260806 22:28:15.936828 16636 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34163
I20260806 22:28:15.937148 16636 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:15.937871 16636 heartbeater.cc:511] Master 127.1.8.126:34163 requested a full tablet report, sending...
I20260806 22:28:15.939313 16137 ts_manager.cc:194] Registered new tserver with Master: 2dfa269051ef4a818575b88b6e0d797b (127.1.8.66:43471)
I20260806 22:28:15.940443 16137 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.66:43095
W20260806 22:28:16.278439 16641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:16.278702 16641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:16.278869 16641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:16.286172 16641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:16.286263 16641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:16.286425 16641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:28:16.295414 16641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:16.296710 16641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:16.298549 16641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:16.312350 16653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:16.312367 16651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:16.312453 16650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:16.313011 16641 server_base.cc:1034] running on GCE node
I20260806 22:28:16.313684 16641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:16.314646 16641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:16.315878 16641 hybrid_clock.cc:648] HybridClock initialized: now 1786055296315826 us; error 69 us; skew 500 ppm
I20260806 22:28:16.316282 16641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:16.318681 16641 webserver.cc:460] Webserver started at http://127.1.8.67:44447/ using document root <none> and password file <none>
I20260806 22:28:16.319401 16641 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:16.319540 16641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:16.319893 16641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:16.322883 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "8446fb4742f746aa9cac6a3fa2cfcbe5"
format_stamp: "Formatted at 2026-08-06 22:28:16 on dist-test-slave-7vjv"
server_key: "f4d0858d6ac6f88ae9b4c07eb6e84bbc"
server_key_iv: "3738340250d8104bccebf41358b07575"
server_key_version: "encryptionkey@0"
I20260806 22:28:16.323776 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "8446fb4742f746aa9cac6a3fa2cfcbe5"
format_stamp: "Formatted at 2026-08-06 22:28:16 on dist-test-slave-7vjv"
server_key: "f4d0858d6ac6f88ae9b4c07eb6e84bbc"
server_key_iv: "3738340250d8104bccebf41358b07575"
server_key_version: "encryptionkey@0"
I20260806 22:28:16.329052 16641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:28:16.332501 16661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:16.333866 16641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:28:16.334019 16641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "8446fb4742f746aa9cac6a3fa2cfcbe5"
format_stamp: "Formatted at 2026-08-06 22:28:16 on dist-test-slave-7vjv"
server_key: "f4d0858d6ac6f88ae9b4c07eb6e84bbc"
server_key_iv: "3738340250d8104bccebf41358b07575"
server_key_version: "encryptionkey@0"
I20260806 22:28:16.334201 16641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:16.347947 16641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:16.348785 16641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:16.349016 16641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:16.350131 16641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:16.351583 16641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:16.351671 16641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:16.351743 16641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:16.351801 16641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:16.385608 16641 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:33217
I20260806 22:28:16.385689 16846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:33217 every 8 connection(s)
I20260806 22:28:16.386986 16641 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:28:16.396497 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 16641
I20260806 22:28:16.396684 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:28:16.397125 1057 external_mini_cluster.cc:1428] Setting key defaafa740ecd2a0c39eea549cc26196
I20260806 22:28:16.399616 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:16.389673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:58293 (local address 127.1.8.126:34163)
0806 22:28:16.389863 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:16.389867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:16.391194 (+ 1327us) server_negotiation.cc:407] Connection header received
0806 22:28:16.393065 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:16.393070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:16.393137 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:16.393225 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:16.394339 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.395114 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.396695 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.397148 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.397300 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:16.398773 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:16.398778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:16.398781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:16.398797 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:16.398828 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:16.398840 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:16.398933 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:16.399164 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:28:16.399261 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:28:16.400741 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:16.390182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34163 (local address 127.1.8.67:58293)
0806 22:28:16.391048 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:16.391090 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:16.392808 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:16.393419 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:16.393426 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:16.393508 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:16.394169 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:16.394187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.395245 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.395248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:16.396524 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:16.396536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.397396 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.397405 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:16.397663 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:16.398420 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:16.398424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:16.398476 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:16.398576 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:16.398935 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:16.398948 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:16.398968 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 22:28:16.399149 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 22:28:16.399504 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260806 22:28:16.401989 16849 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34163
I20260806 22:28:16.402241 16849 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:16.402909 16849 heartbeater.cc:511] Master 127.1.8.126:34163 requested a full tablet report, sending...
I20260806 22:28:16.404250 16137 ts_manager.cc:194] Registered new tserver with Master: 8446fb4742f746aa9cac6a3fa2cfcbe5 (127.1.8.67:33217)
I20260806 22:28:16.404999 16137 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.67:58293
I20260806 22:28:16.412324 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:28:16.419426 16433 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:16.417661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.1.8.126:34163)
0806 22:28:16.417865 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:16.417873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:16.418523 (+ 650us) server_negotiation.cc:407] Connection header received
0806 22:28:16.418622 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:16.418626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:16.418701 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:16.418787 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:16.419203 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45876: BlockingRecv error: recv got EOF from 127.0.0.1:45876 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:28:16.420269 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 16233
I20260806 22:28:16.428815 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:16.421172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.1.8.65:37335)
0806 22:28:16.421729 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:16.421736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:16.421761 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:28:16.421816 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:16.421821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:16.422029 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:16.422155 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:16.422507 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.423870 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.425828 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.426389 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.426506 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:16.427432 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:16.427441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:16.427452 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:16.428048 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:16.428147 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:16.428173 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:16.428271 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:16.428417 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:28:16.428526 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":282,"threads_started":1}
I20260806 22:28:16.430022 16424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:16.511143 16431 heartbeater.cc:503] Master 127.1.8.126:34163 was elected leader, sending a full tablet report...
I20260806 22:28:16.512440 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 16233
W20260806 22:28:16.537598 16113 connection.cc:535] server connection from 127.1.8.65:53117 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:28:16.537977 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 16435
I20260806 22:28:16.545938 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:16.538930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.1.8.66:43471)
0806 22:28:16.539527 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:16.539534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:16.539562 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:28:16.539636 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:16.539642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:16.539888 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:16.540014 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:16.540560 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.541561 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.542759 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.543313 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.543419 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:16.544548 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:16.544556 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:16.544569 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:16.545200 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:16.545246 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:16.545264 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:16.545354 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:16.545509 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:28:16.545621 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":325,"threads_started":1}
I20260806 22:28:16.547220 16630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:16.629475 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 16435
I20260806 22:28:16.657963 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 16641
I20260806 22:28:16.666510 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:16.659004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33312 (local address 127.1.8.67:33217)
0806 22:28:16.659688 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:16.659696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:16.659728 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:28:16.659789 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:16.659796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:16.660038 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:16.660266 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:16.660740 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.661802 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.663120 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:16.663686 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:16.663784 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:16.664965 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:16.664973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:16.664988 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:16.665625 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:16.665672 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:16.665692 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:16.665791 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:16.665980 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:28:16.666118 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":354,"threads_started":1}
I20260806 22:28:16.667896 16835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:16.750888 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 16641
I20260806 22:28:16.775297 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 16094
I20260806 22:28:16.776741 16184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:16.881779 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 16094
2026-08-06T22:28:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2554 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:28:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:28:16Z Disabled control of system clock
I20260806 22:28:16.991626 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45713
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:28:17.347916 16881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:17.348291 16881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:17.348486 16881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:17.355712 16881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:28:17.355881 16881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:17.355983 16881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:17.356081 16881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:28:17.356200 16881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:28:17.365455 16881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:45713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:17.366834 16881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:17.368947 16881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:17.381616 16891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:17.381675 16886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:17.381641 16888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:17.382402 16881 server_base.cc:1034] running on GCE node
I20260806 22:28:17.383270 16881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:17.384825 16881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:17.386135 16881 hybrid_clock.cc:648] HybridClock initialized: now 1786055297386056 us; error 32 us; skew 500 ppm
I20260806 22:28:17.386571 16881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:17.389312 16881 webserver.cc:460] Webserver started at http://127.1.8.126:38079/ using document root <none> and password file <none>
I20260806 22:28:17.390105 16881 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:17.390250 16881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:17.390645 16881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:17.394001 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "1ad8177becbf44e3aa958d4b3ca04595"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "6713391d4e066e5e60c433074174b73c"
server_key_iv: "963c985ffe610737c452f478f1bd7ae1"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.394903 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "1ad8177becbf44e3aa958d4b3ca04595"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "6713391d4e066e5e60c433074174b73c"
server_key_iv: "963c985ffe610737c452f478f1bd7ae1"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.400743 16881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:28:17.404443 16901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:17.405798 16881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:28:17.406003 16881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "1ad8177becbf44e3aa958d4b3ca04595"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "6713391d4e066e5e60c433074174b73c"
server_key_iv: "963c985ffe610737c452f478f1bd7ae1"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.406270 16881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:17.412602 16881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:17.413422 16881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:17.413748 16881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:17.434154 16881 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:45713
I20260806 22:28:17.434191 16982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:45713 every 8 connection(s)
I20260806 22:28:17.435647 16881 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:28:17.439384 16983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:17.440172 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 16881
I20260806 22:28:17.440398 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:28:17.441169 1057 external_mini_cluster.cc:1428] Setting key 4d391337642c44744aee192d6b5e9d16
I20260806 22:28:17.450460 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:17.443503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.1.8.126:45713)
0806 22:28:17.444709 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:17.444732 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:17.444797 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:28:17.446436 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:17.446478 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:17.447022 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:17.447502 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:17.448507 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:17.448514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:17.448530 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:17.448549 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:17.448625 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:17.448646 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:17.448877 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:17.449024 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:28:17.449290 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":379,"threads_started":1}
I20260806 22:28:17.454378 16983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595: Bootstrap starting.
I20260806 22:28:17.457304 16983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:17.458277 16983 log.cc:824] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595: Log is configured to *not* fsync() on all Append() calls
I20260806 22:28:17.461212 16983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595: No bootstrap required, opened a new log
I20260806 22:28:17.468815 16983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } }
I20260806 22:28:17.469304 16983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:17.469432 16983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad8177becbf44e3aa958d4b3ca04595, State: Initialized, Role: FOLLOWER
I20260806 22:28:17.470002 16983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } }
I20260806 22:28:17.470242 16983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:28:17.470392 16983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:28:17.470577 16983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:17.472769 16983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } }
I20260806 22:28:17.473301 16983 leader_election.cc:302] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ad8177becbf44e3aa958d4b3ca04595; no voters:
I20260806 22:28:17.473980 16983 leader_election.cc:288] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:28:17.474118 16991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:28:17.474881 16991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [term 1 LEADER]: Becoming Leader. State: Replica: 1ad8177becbf44e3aa958d4b3ca04595, State: Running, Role: LEADER
I20260806 22:28:17.475440 16991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } }
I20260806 22:28:17.476197 16983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:28:17.478217 16992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ad8177becbf44e3aa958d4b3ca04595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } } }
I20260806 22:28:17.478379 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ad8177becbf44e3aa958d4b3ca04595. Latest consensus state: current_term: 1 leader_uuid: "1ad8177becbf44e3aa958d4b3ca04595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad8177becbf44e3aa958d4b3ca04595" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45713 } } }
I20260806 22:28:17.478641 16992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:17.478672 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595 [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:17.479692 17000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:28:17.488865 17000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:28:17.498862 17000 catalog_manager.cc:1347] Generated new cluster ID: c373b05d204c49e49fbef4781ecea2cb
I20260806 22:28:17.499027 17000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:28:17.516611 17000 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:28:17.517073 17000 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:28:17.525244 17000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ad8177becbf44e3aa958d4b3ca04595: Generated new TSK 0
I20260806 22:28:17.526142 17000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:28:17.534265 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--builtin_ntp_servers=127.1.8.84:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:28:17.911378 17024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:17.911707 17024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:17.911880 17024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:17.919203 17024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:17.919291 17024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:17.919497 17024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:28:17.928290 17024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:17.929538 17024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:17.931294 17024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:17.944422 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:17.944453 17030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:17.944516 17037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:17.945247 17024 server_base.cc:1034] running on GCE node
I20260806 22:28:17.945896 17024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:17.946870 17024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:17.948141 17024 hybrid_clock.cc:648] HybridClock initialized: now 1786055297948058 us; error 70 us; skew 500 ppm
I20260806 22:28:17.948487 17024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:17.950795 17024 webserver.cc:460] Webserver started at http://127.1.8.65:42195/ using document root <none> and password file <none>
I20260806 22:28:17.951519 17024 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:17.951655 17024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:17.952003 17024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:17.955097 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "1f486fb8f6f0426fa5c378ba6fd008ce"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "0712e945aeee7da29b9949b2ac5295ef"
server_key_iv: "e67ab8a304e70044275e55c01f975f0d"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.955893 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "1f486fb8f6f0426fa5c378ba6fd008ce"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "0712e945aeee7da29b9949b2ac5295ef"
server_key_iv: "e67ab8a304e70044275e55c01f975f0d"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.961218 17024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 22:28:17.965698 17044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:17.968251 17024 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.004s
I20260806 22:28:17.968508 17024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "1f486fb8f6f0426fa5c378ba6fd008ce"
format_stamp: "Formatted at 2026-08-06 22:28:17 on dist-test-slave-7vjv"
server_key: "0712e945aeee7da29b9949b2ac5295ef"
server_key_iv: "e67ab8a304e70044275e55c01f975f0d"
server_key_version: "encryptionkey@0"
I20260806 22:28:17.968820 17024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:17.975216 17024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:17.976080 17024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:17.976465 17024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:17.977576 17024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:17.979084 17024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:17.979210 17024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:17.979350 17024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:17.979439 17024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.013821 17024 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44385
I20260806 22:28:18.013880 17224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44385 every 8 connection(s)
I20260806 22:28:18.015214 17024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:28:18.021973 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 17024
I20260806 22:28:18.022228 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:28:18.022908 1057 external_mini_cluster.cc:1428] Setting key 2d38c36f84c45788b1b363988678bfc5
I20260806 22:28:18.023823 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.017811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:38829 (local address 127.1.8.126:45713)
0806 22:28:18.018315 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:18.018319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:18.019350 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 22:28:18.021248 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:18.021253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:18.021334 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:18.021462 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:18.022939 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:18.022946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:18.022951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:18.022972 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:18.023049 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:18.023066 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:18.023198 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:18.023396 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:28:18.023481 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":275,"threads_started":1}
I20260806 22:28:18.025652 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.018294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45713 (local address 127.1.8.65:38829)
0806 22:28:18.019190 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:18.019226 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:18.020957 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:18.021751 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:18.021760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:18.021858 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:18.022625 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:18.022627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:18.022668 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:18.022750 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:18.023157 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:18.023171 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:18.023180 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:28:18.023479 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 22:28:18.023759 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":275,"threads_started":1}
I20260806 22:28:18.026011 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--builtin_ntp_servers=127.1.8.84:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:28:18.027289 17225 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45713
I20260806 22:28:18.027618 17225 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:18.028312 17225 heartbeater.cc:511] Master 127.1.8.126:45713 requested a full tablet report, sending...
I20260806 22:28:18.030447 16925 ts_manager.cc:194] Registered new tserver with Master: 1f486fb8f6f0426fa5c378ba6fd008ce (127.1.8.65:44385)
W20260806 22:28:18.369199 17231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:18.369455 17231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:18.369629 17231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:18.376849 17231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:18.376933 17231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:18.377135 17231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:28:18.386066 17231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:18.387247 17231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:18.388993 17231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:18.401916 17243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:18.401856 17237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:18.401954 17238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:18.403029 17231 server_base.cc:1034] running on GCE node
I20260806 22:28:18.403679 17231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:18.404649 17231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:18.405879 17231 hybrid_clock.cc:648] HybridClock initialized: now 1786055298405841 us; error 50 us; skew 500 ppm
I20260806 22:28:18.406201 17231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:18.408660 17231 webserver.cc:460] Webserver started at http://127.1.8.66:41015/ using document root <none> and password file <none>
I20260806 22:28:18.409350 17231 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:18.409480 17231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:18.409865 17231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:18.412770 17231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "7e9553393822414792e8208b8a5f7519"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "e3dbc642eb2b1a5cae3f4b427d8f5f0e"
server_key_iv: "59e8bf877fc14510743694b574428184"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.413558 17231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "7e9553393822414792e8208b8a5f7519"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "e3dbc642eb2b1a5cae3f4b427d8f5f0e"
server_key_iv: "59e8bf877fc14510743694b574428184"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.418782 17231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:28:18.422281 17254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.423570 17231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:28:18.423780 17231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "7e9553393822414792e8208b8a5f7519"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "e3dbc642eb2b1a5cae3f4b427d8f5f0e"
server_key_iv: "59e8bf877fc14510743694b574428184"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.424031 17231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:18.430632 17231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:18.431372 17231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:18.431607 17231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:18.432543 17231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:18.434059 17231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:18.434149 17231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.434305 17231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:18.434392 17231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.469617 17231 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:45523
I20260806 22:28:18.469672 17441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:45523 every 8 connection(s)
I20260806 22:28:18.470935 17231 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:28:18.477191 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 17231
I20260806 22:28:18.477398 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:28:18.477877 1057 external_mini_cluster.cc:1428] Setting key c9f1ec68c10130768415616857a57524
I20260806 22:28:18.479573 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.473591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:57901 (local address 127.1.8.126:45713)
0806 22:28:18.473768 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:18.473773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:18.475092 (+ 1319us) server_negotiation.cc:407] Connection header received
0806 22:28:18.476855 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:18.476861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:18.476939 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:18.477054 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:18.478614 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:18.478622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:18.478626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:18.478650 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:18.478697 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:18.478717 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:18.478870 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:18.479111 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 22:28:18.479183 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:28:18.480489 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.474041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45713 (local address 127.1.8.66:57901)
0806 22:28:18.474937 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:18.474974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:18.476583 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:18.477297 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:18.477309 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:18.477418 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:18.478197 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:18.478201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:18.478251 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:18.478358 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:18.478843 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:18.478856 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:18.478863 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:28:18.479149 (+ 286us) client_negotiation.cc:240] Negotiation successful
0806 22:28:18.479336 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":264,"threads_started":1}
I20260806 22:28:18.481662 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--builtin_ntp_servers=127.1.8.84:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:28:18.482151 17442 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45713
I20260806 22:28:18.482452 17442 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:18.483040 17442 heartbeater.cc:511] Master 127.1.8.126:45713 requested a full tablet report, sending...
I20260806 22:28:18.484575 16925 ts_manager.cc:194] Registered new tserver with Master: 7e9553393822414792e8208b8a5f7519 (127.1.8.66:45523)
W20260806 22:28:18.829319 17446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:18.829624 17446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:18.829761 17446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:18.836793 17446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:18.836886 17446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:18.837028 17446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:28:18.845539 17446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:18.846709 17446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:18.848418 17446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:18.861398 17452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:18.861436 17455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:18.861490 17453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:18.862279 17446 server_base.cc:1034] running on GCE node
I20260806 22:28:18.862876 17446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:18.863811 17446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:18.865020 17446 hybrid_clock.cc:648] HybridClock initialized: now 1786055298864963 us; error 56 us; skew 500 ppm
I20260806 22:28:18.865324 17446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:28:18.867398 17446 webserver.cc:460] Webserver started at http://127.1.8.67:39247/ using document root <none> and password file <none>
I20260806 22:28:18.868054 17446 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:18.868235 17446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:18.868558 17446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:18.871170 17446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "4845c6a360a740ac8c77da547e95e32c"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "dc4ba1864ab5f32c94513c3da691e869"
server_key_iv: "be3f6c477b872c7106dd9cc9c053bf6d"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.871948 17446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "4845c6a360a740ac8c77da547e95e32c"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "dc4ba1864ab5f32c94513c3da691e869"
server_key_iv: "be3f6c477b872c7106dd9cc9c053bf6d"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.876869 17446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:28:18.880324 17468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.881677 17446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:28:18.881856 17446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "4845c6a360a740ac8c77da547e95e32c"
format_stamp: "Formatted at 2026-08-06 22:28:18 on dist-test-slave-7vjv"
server_key: "dc4ba1864ab5f32c94513c3da691e869"
server_key_iv: "be3f6c477b872c7106dd9cc9c053bf6d"
server_key_version: "encryptionkey@0"
I20260806 22:28:18.882087 17446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:18.887571 17446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:18.888417 17446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:18.888695 17446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:18.889583 17446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:18.890965 17446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:18.891100 17446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.891228 17446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:18.891319 17446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:18.925325 17446 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:35297
I20260806 22:28:18.925420 17642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:35297 every 8 connection(s)
I20260806 22:28:18.926615 17446 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:28:18.933262 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 17446
I20260806 22:28:18.933435 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:28:18.933899 1057 external_mini_cluster.cc:1428] Setting key f6618bac609fd906be7b16178cbbc243
I20260806 22:28:18.935250 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.929322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:36309 (local address 127.1.8.126:45713)
0806 22:28:18.929501 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:18.929505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:18.930932 (+ 1427us) server_negotiation.cc:407] Connection header received
0806 22:28:18.932722 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:18.932728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:18.932804 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:18.932934 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:18.934348 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:18.934355 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:18.934362 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:18.934384 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:18.934424 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:18.934441 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:18.934567 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:18.934817 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:28:18.934924 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:28:18.936340 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.929796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45713 (local address 127.1.8.67:36309)
0806 22:28:18.930760 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:18.930804 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:18.932454 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:18.933165 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:18.933173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:18.933258 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:18.934053 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:28:18.934055 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:18.934094 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:28:18.934177 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:18.934556 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:18.934566 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:18.934592 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:28:18.934897 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 22:28:18.935132 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":284,"threads_started":1}
I20260806 22:28:18.937556 17644 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45713
I20260806 22:28:18.937848 17644 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:18.938401 17644 heartbeater.cc:511] Master 127.1.8.126:45713 requested a full tablet report, sending...
I20260806 22:28:18.939694 16925 ts_manager.cc:194] Registered new tserver with Master: 4845c6a360a740ac8c77da547e95e32c (127.1.8.67:35297)
I20260806 22:28:18.949887 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:28:18.956457 17228 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:28:18.954734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.1.8.126:45713)
0806 22:28:18.954971 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:18.954979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:18.955388 (+ 409us) server_negotiation.cc:407] Connection header received
0806 22:28:18.955615 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:18.955620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:18.955672 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:18.955802 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:18.956246 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49184: BlockingRecv error: recv got EOF from 127.0.0.1:49184 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:28:18.957290 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 17024
I20260806 22:28:18.960846 17658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:18.958356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.1.8.65:44385)
0806 22:28:18.958865 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:18.958872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:18.958896 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:28:18.958952 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:18.958957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:18.959143 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:18.959259 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:18.960253 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:18.960260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:18.960270 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:18.960284 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:18.960334 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:18.960349 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:18.960450 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:18.960535 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:28:18.960614 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":243,"threads_started":1}
I20260806 22:28:18.961997 17209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:19.042403 17225 heartbeater.cc:503] Master 127.1.8.126:45713 was elected leader, sending a full tablet report...
I20260806 22:28:19.044292 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 17024
I20260806 22:28:19.070873 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 17231
I20260806 22:28:19.074472 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:19.071914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.1.8.66:45523)
0806 22:28:19.072506 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:19.072513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:19.072542 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:28:19.072605 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:19.072610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:19.072826 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:19.072980 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:19.073843 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:19.073850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:19.073861 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:19.073881 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:19.073924 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:19.073937 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:19.074054 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:19.074154 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:28:19.074234 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":307,"threads_started":1}
I20260806 22:28:19.075678 17435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:19.156561 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 17231
I20260806 22:28:19.184320 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 17446
I20260806 22:28:19.188426 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:19.185424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.1.8.67:35297)
0806 22:28:19.186232 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:19.186238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:19.186267 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:28:19.186330 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:19.186336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:19.186551 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:19.186678 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:19.187728 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:19.187734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:19.187745 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:28:19.187757 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:19.187794 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:28:19.187807 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:19.187936 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:19.187988 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:28:19.188074 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":356,"threads_started":1}
I20260806 22:28:19.189631 17635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:19.272150 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 17446
I20260806 22:28:19.296002 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 16881
I20260806 22:28:19.297276 16974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:19.402417 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 16881
2026-08-06T22:28:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2518 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:28:19 dist-test-slave-7vjv krb5kdc[17675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:28:19 dist-test-slave-7vjv krb5kdc[17675](info): set up 2 sockets
Aug 06 22:28:19 dist-test-slave-7vjv krb5kdc[17675](info): commencing operation
krb5kdc: starting...
W20260806 22:28:20.744742 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:28:20 dist-test-slave-7vjv krb5kdc[17675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:28:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:28:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:28:21.043030 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43069
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:43069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:28:21.382609 17697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:21.382867 17697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:21.382982 17697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:21.390194 17697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:28:21.390302 17697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:21.390350 17697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:21.390410 17697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:28:21.390456 17697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:28:21.399961 17697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:43069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:21.401261 17697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:21.402549 17697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:21.414686 17710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:21.414637 17713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:21.414886 17708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:21.415341 17697 server_base.cc:1034] running on GCE node
I20260806 22:28:21.416211 17697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:21.417452 17697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:21.418658 17697 hybrid_clock.cc:648] HybridClock initialized: now 1786055301418614 us; error 32 us; skew 500 ppm
I20260806 22:28:21.418992 17697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:28:21 dist-test-slave-7vjv krb5kdc[17675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055301, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:28:21.425834 17697 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:28:21.427328 17697 webserver.cc:460] Webserver started at http://127.1.8.126:35367/ using document root <none> and password file <none>
I20260806 22:28:21.428015 17697 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:21.428188 17697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:21.428503 17697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:21.431463 17697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "82da47b5bee24dbea9e8b6e7056d64fb"
format_stamp: "Formatted at 2026-08-06 22:28:21 on dist-test-slave-7vjv"
server_key: "7e6f1d34a740381c69ad35f9199a1aba"
server_key_iv: "a146f3c378b7783aafca05307d7b27e6"
server_key_version: "encryptionkey@0"
I20260806 22:28:21.432331 17697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "82da47b5bee24dbea9e8b6e7056d64fb"
format_stamp: "Formatted at 2026-08-06 22:28:21 on dist-test-slave-7vjv"
server_key: "7e6f1d34a740381c69ad35f9199a1aba"
server_key_iv: "a146f3c378b7783aafca05307d7b27e6"
server_key_version: "encryptionkey@0"
I20260806 22:28:21.438304 17697 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:28:21.441751 17722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:21.443068 17697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:28:21.443253 17697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "82da47b5bee24dbea9e8b6e7056d64fb"
format_stamp: "Formatted at 2026-08-06 22:28:21 on dist-test-slave-7vjv"
server_key: "7e6f1d34a740381c69ad35f9199a1aba"
server_key_iv: "a146f3c378b7783aafca05307d7b27e6"
server_key_version: "encryptionkey@0"
I20260806 22:28:21.443457 17697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:21.457450 17697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:21.461943 17697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:21.462251 17697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:21.481763 17697 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:43069
I20260806 22:28:21.481783 17796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:43069 every 8 connection(s)
I20260806 22:28:21.483268 17697 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:28:21.486970 17798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:21.490823 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 17697
I20260806 22:28:21.491050 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:28:21.491917 1057 external_mini_cluster.cc:1428] Setting key 5445371e8d6a123643871fd333b03090
I20260806 22:28:21.500890 17798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: Bootstrap starting.
I20260806 22:28:21.504707 17798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:21.506244 17798 log.cc:824] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:28:21 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:28:21.509203 17798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: No bootstrap required, opened a new log
I20260806 22:28:21.516422 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:21.494078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.1.8.126:43069)
0806 22:28:21.494841 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:21.494853 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:21.494895 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:28:21.496599 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:21.496620 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:21.496995 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:21.497304 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:21.499124 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:21.500543 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:21.502319 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:21.503008 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:21.503139 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:21.508136 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:21.508151 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:21.508173 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:21.508223 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:21.511829 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:21.512675 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:21.512692 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:21.512698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:21.512889 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:21.513322 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:21.513330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:21.513339 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:21.513931 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:21.514308 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:21.514786 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 22:28:21.515231 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":383,"threads_started":1}
I20260806 22:28:21.517081 17798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } }
I20260806 22:28:21.517441 17798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:21.517613 17798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82da47b5bee24dbea9e8b6e7056d64fb, State: Initialized, Role: FOLLOWER
I20260806 22:28:21.518203 17798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } }
I20260806 22:28:21.518457 17798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:28:21.518613 17798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:28:21.518793 17798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:21.521246 17798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } }
I20260806 22:28:21.521727 17798 leader_election.cc:302] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82da47b5bee24dbea9e8b6e7056d64fb; no voters:
I20260806 22:28:21.522341 17798 leader_election.cc:288] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:28:21.522490 17807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:28:21.523310 17807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [term 1 LEADER]: Becoming Leader. State: Replica: 82da47b5bee24dbea9e8b6e7056d64fb, State: Running, Role: LEADER
I20260806 22:28:21.523916 17807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } }
I20260806 22:28:21.524592 17798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:28:21.526481 17810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82da47b5bee24dbea9e8b6e7056d64fb. Latest consensus state: current_term: 1 leader_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } } }
I20260806 22:28:21.527216 17810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:21.527226 17808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82da47b5bee24dbea9e8b6e7056d64fb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43069 } } }
I20260806 22:28:21.528215 17808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb [sys.catalog]: This master's current role is: LEADER
I20260806 22:28:21.537503 17820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:28:21.543254 17820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:28:21.552464 17820 catalog_manager.cc:1347] Generated new cluster ID: 7beb569718774085877b1b13cf4c435d
I20260806 22:28:21.552642 17820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:28:21.564320 17820 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:28:21.565646 17820 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:28:21.572642 17820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: Generated new TSK 0
I20260806 22:28:21.573464 17820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:28:21.655169 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:28:22.002700 17837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:22.002982 17837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:22.003140 17837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:22.011109 17837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:22.011214 17837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:22.011404 17837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:28:22.021163 17837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:22.022480 17837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:22.023800 17837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:22.037001 17846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:22.037135 17850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:22.036993 17852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:22.038192 17837 server_base.cc:1034] running on GCE node
I20260806 22:28:22.038986 17837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:22.039963 17837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:22.041213 17837 hybrid_clock.cc:648] HybridClock initialized: now 1786055302041190 us; error 68 us; skew 500 ppm
I20260806 22:28:22.041558 17837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:28:22 dist-test-slave-7vjv krb5kdc[17675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055302, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:28:22.044883 17837 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:28:22.046411 17837 webserver.cc:460] Webserver started at http://127.1.8.65:44927/ using document root <none> and password file <none>
I20260806 22:28:22.047174 17837 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:22.047319 17837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:22.047623 17837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:22.050673 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "fe933c9552a44e898ad8e324df482552"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "893e038509595cf1276a33b56285c801"
server_key_iv: "754f6af2590b2694cfb14f51b3f69117"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.051482 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "fe933c9552a44e898ad8e324df482552"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "893e038509595cf1276a33b56285c801"
server_key_iv: "754f6af2590b2694cfb14f51b3f69117"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.056833 17837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 22:28:22.060165 17865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.061502 17837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:28:22.061685 17837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "fe933c9552a44e898ad8e324df482552"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "893e038509595cf1276a33b56285c801"
server_key_iv: "754f6af2590b2694cfb14f51b3f69117"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.061894 17837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:22.079212 17837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:22.083118 17837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:22.083406 17837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:22.084434 17837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:22.085831 17837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:22.085956 17837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.086115 17837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:22.086191 17837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.120713 17837 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:35693
I20260806 22:28:22.120808 18039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:35693 every 8 connection(s)
I20260806 22:28:22.122051 17837 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:28:22.125897 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 17837
I20260806 22:28:22.126142 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:28:22.126924 1057 external_mini_cluster.cc:1428] Setting key a31429af237376db0d40199f48afe22b
Aug 06 22:28:22 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055302, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:28:22.143085 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:22.124710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:50167 (local address 127.1.8.126:43069)
0806 22:28:22.125361 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:22.125366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:22.126409 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 22:28:22.128580 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:22.128586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:22.128685 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:22.128808 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:22.130309 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.131046 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.132702 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.133097 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.133238 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:22.137136 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:22.137142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:22.137147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:22.137189 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:22.139938 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:22.140702 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:22.140708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:22.140712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:22.140843 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:22.141303 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:22.141309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:22.141312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:22.141585 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:22.141748 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:22.142465 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 22:28:22.142685 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":346,"threads_started":1}
I20260806 22:28:22.144014 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:22.125274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43069 (local address 127.1.8.65:50167)
0806 22:28:22.126234 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:22.126281 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:22.128299 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:22.129074 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:22.129082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:22.129585 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:22.130137 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:22.130153 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.131231 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.131235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:22.132552 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:22.132564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.133343 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.133350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:22.133581 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:22.134384 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:22.134430 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:22.136908 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:22.140114 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:22.140123 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:22.140141 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:22.140566 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:22.140969 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:22.140975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:22.140979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:22.141178 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:22.141722 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:22.141729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:22.142070 (+ 341us) client_negotiation.cc:769] Sending connection context
0806 22:28:22.142348 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 22:28:22.142873 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":430,"threads_started":1}
I20260806 22:28:22.145637 18040 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43069
I20260806 22:28:22.145995 18040 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:22.147038 18040 heartbeater.cc:511] Master 127.1.8.126:43069 requested a full tablet report, sending...
I20260806 22:28:22.149318 17746 ts_manager.cc:194] Registered new tserver with Master: fe933c9552a44e898ad8e324df482552 (127.1.8.65:35693)
I20260806 22:28:22.150739 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:50167
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:28:22.199522 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:28:22.549750 18049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:22.550027 18049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:22.550266 18049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:22.557317 18049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:22.557413 18049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:22.557613 18049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:28:22.567991 18049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:22.569504 18049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:22.571025 18049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:22.584589 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:22.584656 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:22.584862 18061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:22.586094 18049 server_base.cc:1034] running on GCE node
I20260806 22:28:22.586699 18049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:22.587638 18049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:22.588861 18049 hybrid_clock.cc:648] HybridClock initialized: now 1786055302588818 us; error 71 us; skew 500 ppm
I20260806 22:28:22.589210 18049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:28:22 dist-test-slave-7vjv krb5kdc[17675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055302, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:28:22.592518 18049 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:28:22.593974 18049 webserver.cc:460] Webserver started at http://127.1.8.66:39319/ using document root <none> and password file <none>
I20260806 22:28:22.594651 18049 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:22.594803 18049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:22.595127 18049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:22.597944 18049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "2450d8d3ff6c49cb993c4a6e8b41c167"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "31ed57222def0ebd631d9059ca102aae"
server_key_iv: "3936a02e9371cd0f142f857aedec82ec"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.598765 18049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "2450d8d3ff6c49cb993c4a6e8b41c167"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "31ed57222def0ebd631d9059ca102aae"
server_key_iv: "3936a02e9371cd0f142f857aedec82ec"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.603720 18049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:28:22.607072 18072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.608614 18049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:28:22.608785 18049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "2450d8d3ff6c49cb993c4a6e8b41c167"
format_stamp: "Formatted at 2026-08-06 22:28:22 on dist-test-slave-7vjv"
server_key: "31ed57222def0ebd631d9059ca102aae"
server_key_iv: "3936a02e9371cd0f142f857aedec82ec"
server_key_version: "encryptionkey@0"
I20260806 22:28:22.609018 18049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:22.625481 18049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:22.629575 18049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:22.629858 18049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:22.630769 18049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:22.632156 18049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:22.632264 18049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.632424 18049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:22.632515 18049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:22.666960 18049 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:46617
I20260806 22:28:22.667024 18237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:46617 every 8 connection(s)
I20260806 22:28:22.668332 18049 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:28:22.669914 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 18049
I20260806 22:28:22.670101 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:28:22.670619 1057 external_mini_cluster.cc:1428] Setting key 1bc77d0807c524974937ba73e03a0084
Aug 06 22:28:22 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055302, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:28:22.692404 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:22.670961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:42691 (local address 127.1.8.126:43069)
0806 22:28:22.671472 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:22.671481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:22.672552 (+ 1071us) server_negotiation.cc:407] Connection header received
0806 22:28:22.675228 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:22.675233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:22.675314 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:22.675434 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:22.677484 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.678204 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.680707 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.681199 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.681312 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:22.685578 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:22.685583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:22.685587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:22.685621 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:22.688964 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:22.689755 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:22.689761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:22.689765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:22.689938 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:22.690431 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:22.690437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:22.690447 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:22.690723 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:22.690894 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:22.691664 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 22:28:22.691907 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260806 22:28:22.693413 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:22.671533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43069 (local address 127.1.8.66:42691)
0806 22:28:22.672379 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:22.672446 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:22.674907 (+ 2461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:22.675705 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:22.675717 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:22.676525 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:22.677287 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:22.677316 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.678374 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.678382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:22.680515 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:22.680534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:22.681442 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:22.681452 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:22.681765 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:22.682516 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:22.682555 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:22.685311 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:22.689130 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:22.689140 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:22.689158 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:22.689608 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:22.690064 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:22.690069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:22.690073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:22.690292 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:22.690876 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:22.690884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:22.691268 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 22:28:22.691570 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 22:28:22.692162 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":315,"threads_started":1}
I20260806 22:28:22.695217 18238 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43069
I20260806 22:28:22.695559 18238 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:22.696471 18238 heartbeater.cc:511] Master 127.1.8.126:43069 requested a full tablet report, sending...
I20260806 22:28:22.698207 17746 ts_manager.cc:194] Registered new tserver with Master: 2450d8d3ff6c49cb993c4a6e8b41c167 (127.1.8.66:46617)
I20260806 22:28:22.699654 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:42691
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:28:22.741285 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:28:23.077781 18245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:28:23.078060 18245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:28:23.078154 18245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:28:23.085227 18245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:28:23.085330 18245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:28:23.085439 18245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:28:23.094763 18245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:28:23.096082 18245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:28:23.097424 18245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:28:23.111555 18254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:23.111922 18255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:28:23.112187 18258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:28:23.113647 18245 server_base.cc:1034] running on GCE node
I20260806 22:28:23.114295 18245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:28:23.115255 18245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:28:23.116518 18245 hybrid_clock.cc:648] HybridClock initialized: now 1786055303116474 us; error 61 us; skew 500 ppm
I20260806 22:28:23.116851 18245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:28:23 dist-test-slave-7vjv krb5kdc[17675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055303, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:28:23.120373 18245 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:28:23.122119 18245 webserver.cc:460] Webserver started at http://127.1.8.67:37739/ using document root <none> and password file <none>
I20260806 22:28:23.122995 18245 fs_manager.cc:359] Metadata directory not provided
I20260806 22:28:23.123138 18245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:28:23.123499 18245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:28:23.127458 18245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "ac38b4f267664499a1ff438a1a979387"
format_stamp: "Formatted at 2026-08-06 22:28:23 on dist-test-slave-7vjv"
server_key: "f949830a4b369b09fafa3649e4787918"
server_key_iv: "40f48c0d4d82ad40bfe2f502dd7a300b"
server_key_version: "encryptionkey@0"
I20260806 22:28:23.128338 18245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "ac38b4f267664499a1ff438a1a979387"
format_stamp: "Formatted at 2026-08-06 22:28:23 on dist-test-slave-7vjv"
server_key: "f949830a4b369b09fafa3649e4787918"
server_key_iv: "40f48c0d4d82ad40bfe2f502dd7a300b"
server_key_version: "encryptionkey@0"
I20260806 22:28:23.133463 18245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 22:28:23.136897 18269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:23.138181 18245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:28:23.138409 18245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "ac38b4f267664499a1ff438a1a979387"
format_stamp: "Formatted at 2026-08-06 22:28:23 on dist-test-slave-7vjv"
server_key: "f949830a4b369b09fafa3649e4787918"
server_key_iv: "40f48c0d4d82ad40bfe2f502dd7a300b"
server_key_version: "encryptionkey@0"
I20260806 22:28:23.138687 18245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:28:23.154567 18040 heartbeater.cc:503] Master 127.1.8.126:43069 was elected leader, sending a full tablet report...
I20260806 22:28:23.161162 18245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:28:23.165336 18245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:28:23.165647 18245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:28:23.166599 18245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:28:23.167953 18245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:28:23.168030 18245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:23.168221 18245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:28:23.168303 18245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:28:23.202943 18245 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:35751
I20260806 22:28:23.203034 18432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:35751 every 8 connection(s)
I20260806 22:28:23.204366 18245 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:28:23.210057 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 18245
I20260806 22:28:23.210242 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:28:23.210704 1057 external_mini_cluster.cc:1428] Setting key d363a920611cb123d0d01c63ce525332
Aug 06 22:28:23 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055303, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:28:23.225221 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.207040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:54755 (local address 127.1.8.126:43069)
0806 22:28:23.207512 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.207517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.208428 (+ 911us) server_negotiation.cc:407] Connection header received
0806 22:28:23.210569 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.210575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.210691 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.210817 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:23.212625 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.213807 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.215335 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.215727 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.215857 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.219411 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:23.219417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:23.219422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:23.219473 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:23.221972 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.222710 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.222718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.222726 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.222848 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.223307 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.223318 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.223320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.223584 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:23.223747 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.224527 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.224825 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260806 22:28:23.226425 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.207510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43069 (local address 127.1.8.67:54755)
0806 22:28:23.208271 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.208308 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.210256 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.211076 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.211084 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.211583 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:28:23.212455 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.212476 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.213971 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.213977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.215177 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.215194 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.215967 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.215975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.216291 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.216922 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:28:23.216944 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:28:23.219185 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:28:23.222122 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:23.222131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:23.222147 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:23.222577 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:23.222964 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:28:23.222969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:28:23.222972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:28:23.223181 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:28:23.223717 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:28:23.223725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:28:23.224079 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.224408 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.225013 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260806 22:28:23.227730 18434 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43069
I20260806 22:28:23.227998 18434 heartbeater.cc:464] Registering TS with master...
I20260806 22:28:23.228734 18434 heartbeater.cc:511] Master 127.1.8.126:43069 requested a full tablet report, sending...
I20260806 22:28:23.230069 17746 ts_manager.cc:194] Registered new tserver with Master: ac38b4f267664499a1ff438a1a979387 (127.1.8.67:35751)
I20260806 22:28:23.230854 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:54755
I20260806 22:28:23.238432 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:28:23.255431 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.243222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.1.8.126:43069)
0806 22:28:23.243405 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.243412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.243897 (+ 485us) server_negotiation.cc:407] Connection header received
0806 22:28:23.243992 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.243997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.244085 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.244269 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:23.245645 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.246460 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.247728 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.248179 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.248272 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.250559 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:23.250564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:23.250567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:23.250596 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:23.253116 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.253796 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.253800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.253802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.253918 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.254243 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.254246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.254248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.254552 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:23.254722 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.255035 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.255202 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:28:23.259617 17746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:28:23.266577 17746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:28:23.290930 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.280237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.1.8.65:35693)
0806 22:28:23.282496 (+ 2259us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.282505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.282536 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:28:23.282661 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.282672 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.282948 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.283699 (+ 751us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:28:23.284038 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.285653 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.288490 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.289687 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.289850 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.290082 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.290386 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.290549 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2068,"thread_start_us":2410,"threads_started":1}
I20260806 22:28:23.290979 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.279679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54914)
0806 22:28:23.281501 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.281540 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.281669 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.283235 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.283244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.283278 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:28:23.283867 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.283879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.285796 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.285804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.288295 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.288313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.289909 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.289915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.290125 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.290178 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.290335 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.290523 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"mutex_wait_us":28,"thread_start_us":1537,"threads_started":1}
I20260806 22:28:23.291080 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.280494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:46617 (local address 127.0.0.1:59804)
0806 22:28:23.282346 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.282363 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.282607 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.283551 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.283555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.283580 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:28:23.284094 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.284124 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.286278 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.286286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.287959 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.287974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.290151 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.290156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.290349 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.290385 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.290470 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.290607 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"mutex_wait_us":19,"thread_start_us":1558,"threads_started":1}
I20260806 22:28:23.291826 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.279874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.1.8.66:46617)
0806 22:28:23.281075 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.281086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.282470 (+ 1384us) server_negotiation.cc:407] Connection header received
0806 22:28:23.282826 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.282834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.283107 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.283302 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:28:23.284322 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.286153 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.288194 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.289693 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.289813 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.290004 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.291314 (+ 1310us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.291495 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":932,"threads_started":1}
I20260806 22:28:23.294970 18143 tablet_service.cc:1467] Processing CreateTablet for tablet ec8c981c6e994f009375ca19621eede8 (DEFAULT_TABLE table=test-table [id=8c6e84b3c0684de7ac9f5c1a82e45af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:28:23.294962 17936 tablet_service.cc:1467] Processing CreateTablet for tablet ec8c981c6e994f009375ca19621eede8 (DEFAULT_TABLE table=test-table [id=8c6e84b3c0684de7ac9f5c1a82e45af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:28:23.295930 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.282474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:35751 (local address 127.0.0.1:56222)
0806 22:28:23.283249 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.283267 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.283373 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.285450 (+ 2077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.285455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.285491 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:28:23.285878 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.285889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.287887 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.287893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.290013 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.290032 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.292194 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.292200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.292360 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.292391 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.292478 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.295615 (+ 3137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":767,"threads_started":1}
I20260806 22:28:23.296255 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.282927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.1.8.67:35751)
0806 22:28:23.284597 (+ 1670us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.284606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.284642 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:28:23.284737 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.284745 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.284996 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.285172 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:28:23.286078 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.287696 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.290204 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.291646 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.291772 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.292019 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.295726 (+ 3707us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.295901 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1500,"thread_start_us":1332,"threads_started":1}
I20260806 22:28:23.296389 18143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8c981c6e994f009375ca19621eede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:23.296401 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8c981c6e994f009375ca19621eede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:23.299937 18345 tablet_service.cc:1467] Processing CreateTablet for tablet ec8c981c6e994f009375ca19621eede8 (DEFAULT_TABLE table=test-table [id=8c6e84b3c0684de7ac9f5c1a82e45af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:28:23.301375 18345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8c981c6e994f009375ca19621eede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:28:23.308036 18470 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Bootstrap starting.
I20260806 22:28:23.311619 18470 tablet_bootstrap.cc:654] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:23.312012 18471 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Bootstrap starting.
I20260806 22:28:23.313270 18470 log.cc:824] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Log is configured to *not* fsync() on all Append() calls
I20260806 22:28:23.315589 18471 tablet_bootstrap.cc:654] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:23.316636 18470 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: No bootstrap required, opened a new log
I20260806 22:28:23.316864 18471 log.cc:824] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Log is configured to *not* fsync() on all Append() calls
I20260806 22:28:23.316918 18470 ts_tablet_manager.cc:1397] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 22:28:23.316921 18473 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Bootstrap starting.
I20260806 22:28:23.320560 18473 tablet_bootstrap.cc:654] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Neither blocks nor log segments found. Creating new log.
I20260806 22:28:23.320575 18471 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: No bootstrap required, opened a new log
I20260806 22:28:23.320853 18471 ts_tablet_manager.cc:1397] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 22:28:23.321591 18473 log.cc:824] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Log is configured to *not* fsync() on all Append() calls
I20260806 22:28:23.324019 18473 tablet_bootstrap.cc:492] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: No bootstrap required, opened a new log
I20260806 22:28:23.324309 18473 ts_tablet_manager.cc:1397] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 22:28:23.327427 18470 raft_consensus.cc:348] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.327867 18470 raft_consensus.cc:374] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:23.327960 18470 raft_consensus.cc:729] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe933c9552a44e898ad8e324df482552, State: Initialized, Role: FOLLOWER
I20260806 22:28:23.328615 18470 consensus_queue.cc:260] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.330336 18470 ts_tablet_manager.cc:1428] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 22:28:23.332290 18471 raft_consensus.cc:348] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.332720 18471 raft_consensus.cc:374] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:23.332830 18471 raft_consensus.cc:729] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2450d8d3ff6c49cb993c4a6e8b41c167, State: Initialized, Role: FOLLOWER
I20260806 22:28:23.333446 18471 consensus_queue.cc:260] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.333729 18473 raft_consensus.cc:348] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.334187 18473 raft_consensus.cc:374] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:28:23.334291 18473 raft_consensus.cc:729] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac38b4f267664499a1ff438a1a979387, State: Initialized, Role: FOLLOWER
I20260806 22:28:23.334753 18238 heartbeater.cc:503] Master 127.1.8.126:43069 was elected leader, sending a full tablet report...
I20260806 22:28:23.334983 18473 consensus_queue.cc:260] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.335258 18471 ts_tablet_manager.cc:1428] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260806 22:28:23.336130 18434 heartbeater.cc:503] Master 127.1.8.126:43069 was elected leader, sending a full tablet report...
I20260806 22:28:23.336753 18473 ts_tablet_manager.cc:1428] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
W20260806 22:28:23.374584 18041 tablet.cc:2369] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:28:23.396948 18482 raft_consensus.cc:482] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:28:23.397336 18482 raft_consensus.cc:504] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.399071 18482 leader_election.cc:288] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2450d8d3ff6c49cb993c4a6e8b41c167 (127.1.8.66:46617), fe933c9552a44e898ad8e324df482552 (127.1.8.65:35693)
I20260806 22:28:23.405449 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.399584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:42839 (local address 127.1.8.66:46617)
0806 22:28:23.399813 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.399819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.399845 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:28:23.400115 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.400123 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.400231 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.400346 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:28:23.400788 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.401699 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.403726 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.404558 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.404708 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.404800 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.405073 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.405192 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:28:23.405460 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.399427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:46617 (local address 127.1.8.67:42839)
0806 22:28:23.399705 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.399730 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.399911 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.400383 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.400388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.400422 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:28:23.400638 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.400652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.401883 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.401887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.403567 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.403580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.404720 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.404725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.404914 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.404938 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.405046 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.405175 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 22:28:23.406317 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.399679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:44999 (local address 127.1.8.65:35693)
0806 22:28:23.399886 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.399893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.400763 (+ 870us) server_negotiation.cc:407] Connection header received
0806 22:28:23.400918 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.400922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.400999 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.401101 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:28:23.401738 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.402710 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.404519 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.405277 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.405439 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.405551 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.405954 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.406084 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:28:23.406361 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.399786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.1.8.67:44999)
0806 22:28:23.400638 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.400656 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:28:23.400780 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:28:23.401157 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:28:23.401161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:28:23.401185 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:28:23.401560 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.401569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.402852 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.402856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.404362 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:28:23.404375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.405492 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.405498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:28:23.405799 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.405822 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:28:23.405925 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:28:23.406049 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":473,"threads_started":1}
I20260806 22:28:23.406427 18173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8c981c6e994f009375ca19621eede8" candidate_uuid: "ac38b4f267664499a1ff438a1a979387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" is_pre_election: true
I20260806 22:28:23.407006 18173 raft_consensus.cc:2454] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac38b4f267664499a1ff438a1a979387 in term 0.
I20260806 22:28:23.407404 17972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8c981c6e994f009375ca19621eede8" candidate_uuid: "ac38b4f267664499a1ff438a1a979387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe933c9552a44e898ad8e324df482552" is_pre_election: true
I20260806 22:28:23.407716 18271 leader_election.cc:302] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2450d8d3ff6c49cb993c4a6e8b41c167, ac38b4f267664499a1ff438a1a979387; no voters:
I20260806 22:28:23.407868 17972 raft_consensus.cc:2454] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac38b4f267664499a1ff438a1a979387 in term 0.
I20260806 22:28:23.408133 18482 raft_consensus.cc:2781] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:28:23.408290 18482 raft_consensus.cc:482] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:28:23.408412 18482 raft_consensus.cc:3037] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:23.410741 18482 raft_consensus.cc:504] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
I20260806 22:28:23.411393 18482 leader_election.cc:288] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [CANDIDATE]: Term 1 election: Requested vote from peers 2450d8d3ff6c49cb993c4a6e8b41c167 (127.1.8.66:46617), fe933c9552a44e898ad8e324df482552 (127.1.8.65:35693)
I20260806 22:28:23.411768 18173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8c981c6e994f009375ca19621eede8" candidate_uuid: "ac38b4f267664499a1ff438a1a979387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167"
I20260806 22:28:23.411808 17972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8c981c6e994f009375ca19621eede8" candidate_uuid: "ac38b4f267664499a1ff438a1a979387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe933c9552a44e898ad8e324df482552"
I20260806 22:28:23.411963 18173 raft_consensus.cc:3037] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:23.412017 17972 raft_consensus.cc:3037] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:28:23.414384 18173 raft_consensus.cc:2454] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac38b4f267664499a1ff438a1a979387 in term 1.
I20260806 22:28:23.414407 17972 raft_consensus.cc:2454] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac38b4f267664499a1ff438a1a979387 in term 1.
I20260806 22:28:23.414742 18271 leader_election.cc:302] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2450d8d3ff6c49cb993c4a6e8b41c167, ac38b4f267664499a1ff438a1a979387; no voters:
I20260806 22:28:23.415021 18482 raft_consensus.cc:2781] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:28:23.415505 18482 raft_consensus.cc:686] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 1 LEADER]: Becoming Leader. State: Replica: ac38b4f267664499a1ff438a1a979387, State: Running, Role: LEADER
I20260806 22:28:23.415942 18482 consensus_queue.cc:237] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } }
W20260806 22:28:23.420310 18239 tablet.cc:2369] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:28:23.420374 17745 catalog_manager.cc:5495] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac38b4f267664499a1ff438a1a979387 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "ac38b4f267664499a1ff438a1a979387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac38b4f267664499a1ff438a1a979387" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260806 22:28:23.446633 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.440189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.1.8.67:35751)
0806 22:28:23.440406 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.440413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.440532 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:28:23.440767 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.440772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.440901 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.441054 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:28:23.441467 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.442559 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.444305 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.444953 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.445100 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.445626 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:28:23.446138 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:28:23.446241 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.446301 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.446364 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:28:23.455076 18173 raft_consensus.cc:1261] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 1 FOLLOWER]: Refusing update from remote peer ac38b4f267664499a1ff438a1a979387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:28:23.455075 17972 raft_consensus.cc:1261] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 1 FOLLOWER]: Refusing update from remote peer ac38b4f267664499a1ff438a1a979387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 22:28:23.455976 18436 tablet.cc:2369] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:28:23.456280 18482 consensus_queue.cc:1035] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2450d8d3ff6c49cb993c4a6e8b41c167" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:28:23.456840 18484 consensus_queue.cc:1035] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe933c9552a44e898ad8e324df482552" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 35693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:28:23.484869 17746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:28:23.485208 17746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:28:23.489435 17746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82da47b5bee24dbea9e8b6e7056d64fb: Sending DeleteTablet for 3 replicas of tablet ec8c981c6e994f009375ca19621eede8
I20260806 22:28:23.490890 18143 tablet_service.cc:1514] Processing DeleteTablet for tablet ec8c981c6e994f009375ca19621eede8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:28:23 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:59804
I20260806 22:28:23.490900 18345 tablet_service.cc:1514] Processing DeleteTablet for tablet ec8c981c6e994f009375ca19621eede8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:28:23 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:56222
I20260806 22:28:23.492633 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet ec8c981c6e994f009375ca19621eede8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:28:23 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:54914
I20260806 22:28:23.494601 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 17837
I20260806 22:28:23.516243 18503 tablet_replica.cc:330] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: stopping tablet replica
I20260806 22:28:23.517009 18502 tablet_replica.cc:330] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: stopping tablet replica
I20260806 22:28:23.517642 18503 raft_consensus.cc:2229] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:28:23.518258 18502 raft_consensus.cc:2229] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:28:23.518970 18503 raft_consensus.cc:2258] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:28:23.519819 18502 raft_consensus.cc:2258] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:28:23.522588 18502 ts_tablet_manager.cc:1905] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:28:23 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:28:23.526383 18503 ts_tablet_manager.cc:1905] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:28:23.531384 18502 ts_tablet_manager.cc:1918] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:28:23.531654 18502 log.cc:1196] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/ec8c981c6e994f009375ca19621eede8
I20260806 22:28:23.532413 18502 ts_tablet_manager.cc:1939] T ec8c981c6e994f009375ca19621eede8 P ac38b4f267664499a1ff438a1a979387: Deleting consensus metadata
I20260806 22:28:23.533504 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.496159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.1.8.65:35693)
0806 22:28:23.496337 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.496343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.510777 (+ 14434us) server_negotiation.cc:407] Connection header received
0806 22:28:23.511129 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.511138 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.511274 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.511366 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:23.513736 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.515291 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.517780 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.518729 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.518855 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.525666 (+ 6811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:23.525680 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:23.525696 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:23.525744 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:23.529786 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.530658 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.530667 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.530670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.530853 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.531370 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.531375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.531378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.531907 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:23.532237 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.532850 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.533096 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:28:23.534507 18501 tablet_replica.cc:330] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: stopping tablet replica
I20260806 22:28:23.534742 17723 catalog_manager.cc:4841] TS ac38b4f267664499a1ff438a1a979387 (127.1.8.67:35751): tablet ec8c981c6e994f009375ca19621eede8 (table test-table [id=8c6e84b3c0684de7ac9f5c1a82e45af8]) successfully deleted
I20260806 22:28:23.534914 18032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:23.534971 18501 raft_consensus.cc:2229] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:28:23.535670 18501 raft_consensus.cc:2258] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:28:23.539281 18501 ts_tablet_manager.cc:1905] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:28:23.546504 18501 ts_tablet_manager.cc:1918] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:28:23.546696 18501 log.cc:1196] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/ec8c981c6e994f009375ca19621eede8
I20260806 22:28:23.547194 18501 ts_tablet_manager.cc:1939] T ec8c981c6e994f009375ca19621eede8 P 2450d8d3ff6c49cb993c4a6e8b41c167: Deleting consensus metadata
I20260806 22:28:23.548838 17723 catalog_manager.cc:4841] TS 2450d8d3ff6c49cb993c4a6e8b41c167 (127.1.8.66:46617): tablet ec8c981c6e994f009375ca19621eede8 (table test-table [id=8c6e84b3c0684de7ac9f5c1a82e45af8]) successfully deleted
I20260806 22:28:23.664809 18503 ts_tablet_manager.cc:1918] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:28:23.665140 18503 log.cc:1196] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/ec8c981c6e994f009375ca19621eede8
I20260806 22:28:23.665654 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 17837
I20260806 22:28:23.665763 18503 ts_tablet_manager.cc:1939] T ec8c981c6e994f009375ca19621eede8 P fe933c9552a44e898ad8e324df482552: Deleting consensus metadata
W20260806 22:28:23.698058 17726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:35693 (error 108)
I20260806 22:28:23.698915 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 18049
Aug 06 22:28:23 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:28:23.715183 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.714630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54918)
0806 22:28:23.714842 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.714995 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 22:28:23.715469 17726 catalog_manager.cc:4584] TS fe933c9552a44e898ad8e324df482552 (127.1.8.65:35693): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec8c981c6e994f009375ca19621eede8: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
I20260806 22:28:23.719795 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.700243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.1.8.66:46617)
0806 22:28:23.700422 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.700428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.700757 (+ 329us) server_negotiation.cc:407] Connection header received
0806 22:28:23.700841 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.700846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.700922 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.701025 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:23.702847 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.704009 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.706336 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.706975 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.707095 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.712736 (+ 5641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:23.712752 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:23.712769 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:23.712828 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:23.716921 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.717562 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.717573 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.717577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.717721 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.718138 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.718143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.718148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.718637 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:23.718927 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.719224 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.719436 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:28:23.720526 18231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:23.769375 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.768840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54934)
0806 22:28:23.769075 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.769189 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 22:28:23.770644 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.770184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54946)
0806 22:28:23.770365 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.770462 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:28:23.811257 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 18049
I20260806 22:28:23.831228 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.830723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54956)
0806 22:28:23.830965 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.831059 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:28:23.832437 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.832021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54960)
0806 22:28:23.832204 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.832309 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:28:23.840888 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 18245
Aug 06 22:28:23 dist-test-slave-7vjv krb5kdc[17675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:28:23.860116 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.841625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.1.8.67:35751)
0806 22:28:23.841800 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:28:23.841804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:28:23.841823 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:28:23.842062 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:28:23.842066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:28:23.842140 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:28:23.842234 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:28:23.844031 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.844991 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.846912 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:28:23.847410 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:28:23.847536 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:28:23.853014 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:28:23.853032 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:28:23.853054 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:28:23.853106 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:28:23.857164 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.857834 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.857844 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.857848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.858048 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:28:23.858405 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:28:23.858410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:28:23.858415 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:28:23.858900 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:28:23.859231 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 22:28:23.859403 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:28:23.859666 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:28:23.860937 18425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:23.913230 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.912731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54974)
0806 22:28:23.912955 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.913053 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:28:23.914321 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:23.913943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54986)
0806 22:28:23.914147 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:23.914215 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 22:28:23.987080 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 18245
I20260806 22:28:24.017796 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 17697
I20260806 22:28:24.019021 17788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:28:24.130648 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:28:24.130155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:35693 (local address 127.0.0.1:54994)
0806 22:28:24.130371 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:28:24.130465 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:28:24.130846 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 17697
2026-08-06T22:28:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4730 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:28:24 dist-test-slave-7vjv krb5kdc[18517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:28:24 dist-test-slave-7vjv krb5kdc[18517](info): set up 2 sockets
Aug 06 22:28:24 dist-test-slave-7vjv krb5kdc[18517](info): commencing operation
krb5kdc: starting...
W20260806 22:28:25.499368 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.006s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:28:25 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:28:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:28:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangeradmin_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangeradmin_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangerlookup_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangerlookup_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
I20260806 22:28:25.953416 1057 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/postgres -l logfile start
2026-08-06 22:28:28.550 UTC [18573] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:28:28.550 UTC [18573] LOG: listening on IPv4 address "127.1.8.84", port 36773
2026-08-06 22:28:28.554 UTC [18573] LOG: listening on Unix socket "/tmp/.s.PGSQL.36773"
2026-08-06 22:28:28.578 UTC [18576] LOG: database system was shut down at 2026-08-06 22:28:27 UTC
2026-08-06 22:28:28.582 UTC [18573] LOG: database system is ready to accept connections
I20260806 22:28:29.759166 1057 mini_postgres.cc:96] Postgres bound to 36773
2026-08-06 22:28:29.786 UTC [18593] FATAL: database "slave" does not exist
127.1.8.84:36773 - accepting connections
I20260806 22:28:29.787189 1057 mini_ranger.cc:161] Starting Ranger...
I20260806 22:28:29.820461 1057 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:28:30.318428 1057 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:28:30.318578 1057 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin
2026-08-06 22:28:30,676 [I] DB FLAVOR :POSTGRES
2026-08-06 22:28:30,676 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:28:30,676 [I] Checking connection..
2026-08-06 22:28:30,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:28:30,924 [I] Checking connection passed.
2026-08-06 22:28:30,925 [I] --------- Verifying version history table ---------
2026-08-06 22:28:30,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:28:31,162 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:28:31,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:28:31,422 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:28:31,422 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:28:31,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:28:31.670 UTC [18701] WARNING: there is no transaction in progress
2026-08-06 22:28:31,679 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:28:31,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:28:31,911 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:28:31,912 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:28:31,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:28:32,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:28:32,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-06 22:28:32,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:28:32,959 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:28:32,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:28:33,226 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:28:33,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:28:33,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:28:33,748 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:28:33,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:28:34.738 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.755 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.771 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.787 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.804 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.823 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.876 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.883 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.896 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.906 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.916 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.929 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.940 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.946 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.953 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:34.959 UTC [18867] WARNING: there is no transaction in progress
2026-08-06 22:28:35,574 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:28:35,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:28:35,835 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:28:35,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:28:36,101 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:28:36,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:28:36,355 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:28:36,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:28:36,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:28:36,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-06 22:28:37,137 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:28:37,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:28:37,398 [I] DB_PATCHES have already been applied
I20260806 22:28:37.404500 1057 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksNPSnJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:28:37.404642 1057 mini_ranger.cc:193] Using host: 127.1.8.84
I20260806 22:28:37.428067 1057 mini_ranger.cc:227] Ranger admin URL: http://127.1.8.84:40731
Aug 06, 2026 10:28:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:28:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:28:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:28:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:28:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:28:38.828884 1057 mini_ranger.cc:160] Time spent starting Ranger: real 9.042s user 0.005s sys 0.066s
Aug 06, 2026 10:28:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:28:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:29:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:29:01.048911 1057 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangerkms_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/rangerkms_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:29:01 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055341, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:29:01.288558 1057 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:29:01.321641 1057 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:29:02.182324 1057 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:29:02.182444 1057 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms
I20260806 22:29:02.182622 1057 mini_ranger_kms.cc:227] Using postgres at 127.1.8.84:36773
2026-08-06 22:29:03,035 [I] DB FLAVOR :POSTGRES
2026-08-06 22:29:03,035 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:29:03,035 [I] Checking connection
2026-08-06 22:29:03,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:29:03,312 [I] connection success
2026-08-06 22:29:03,312 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:29:03,312 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:29:03,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:29:03,553 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:29:03,554 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:29:03,554 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:29:03,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:36773/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:29:03,844 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:29:04.320441 1057 mini_ranger_kms.cc:310] Created kms service
I20260806 22:29:04.533991 1057 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:29:04.584237 1057 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:29:04 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055341, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:05.008085 1057 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:29:05.009598 1057 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:29:05.009681 1057 mini_ranger_kms.cc:240] Using host: 127.1.8.84
I20260806 22:29:05.044348 1057 mini_ranger_kms.cc:276] Ranger KMS PID: 19142
I20260806 22:29:05.044554 1057 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.8.84:55009
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:29:07 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055347, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:29:07.889540 1057 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.601s user 0.149s sys 0.496s
I20260806 22:29:07.889953 1057 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:29:07.890203 1057 mini_ranger_kms.cc:401] 127.1.8.84:55009/kms/v1/keys
Aug 06 22:29:08 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055347, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
Aug 06 22:29:08 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055347, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
Aug 06 22:29:11 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:29:11.143795 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42183
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42183
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:55009/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:29:11.690460 19214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:29:11.690860 19214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:29:11.691098 19214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:29:11.699322 19214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:29:11.699545 19214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:29:11.699736 19214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:29:11.699914 19214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:29:11.700040 19214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:29:11.711413 19214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.8.84:55009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42183
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:29:11.712792 19214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:29:11.714242 19214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:29:11.729449 19219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:11.729557 19221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:11.729642 19228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:29:11.730902 19214 server_base.cc:1034] running on GCE node
I20260806 22:29:11.731783 19214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:29:11.733214 19214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:29:11.734459 19214 hybrid_clock.cc:648] HybridClock initialized: now 1786055351734356 us; error 71 us; skew 500 ppm
I20260806 22:29:11.734827 19214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:29:11 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:29:11.738937 19214 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:29:11.740576 19214 webserver.cc:460] Webserver started at http://127.1.8.126:36717/ using document root <none> and password file <none>
I20260806 22:29:11.741197 19214 fs_manager.cc:359] Metadata directory not provided
I20260806 22:29:11.741405 19214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:29:11.741847 19214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:29:11 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:11.800380 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "c871ccb7ff8c4c4cbe57ef5eff362227"
format_stamp: "Formatted at 2026-08-06 22:29:11 on dist-test-slave-7vjv"
server_key: "37671e86ee1b580e1aea100dc7052ac1"
server_key_iv: "778ace274b5be79437769562a0e210b8"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:11.801381 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "c871ccb7ff8c4c4cbe57ef5eff362227"
format_stamp: "Formatted at 2026-08-06 22:29:11 on dist-test-slave-7vjv"
server_key: "37671e86ee1b580e1aea100dc7052ac1"
server_key_iv: "778ace274b5be79437769562a0e210b8"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:11.808368 19214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 22:29:11.828554 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:11.830245 19214 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:29:11.830508 19214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "c871ccb7ff8c4c4cbe57ef5eff362227"
format_stamp: "Formatted at 2026-08-06 22:29:11 on dist-test-slave-7vjv"
server_key: "37671e86ee1b580e1aea100dc7052ac1"
server_key_iv: "778ace274b5be79437769562a0e210b8"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:11.830758 19214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:29:11.846084 19214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:29:11.850787 19214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:29:11.851159 19214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:29:11.884567 19214 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42183
I20260806 22:29:11.884667 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42183 every 8 connection(s)
I20260806 22:29:11.886137 19214 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:29:11.892552 19315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:29:11.894100 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 19214
I20260806 22:29:11.894315 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:29:11.895064 1057 external_mini_cluster.cc:1428] Setting key 1d4d34acc431722430c03a27ed2f00eb
I20260806 22:29:11.908838 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: Bootstrap starting.
Aug 06 22:29:11 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:29:11.913717 19315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: Neither blocks nor log segments found. Creating new log.
I20260806 22:29:11.915179 19315 log.cc:824] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: Log is configured to *not* fsync() on all Append() calls
I20260806 22:29:11.918736 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: No bootstrap required, opened a new log
I20260806 22:29:11.919971 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:11.897353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.1.8.126:42183)
0806 22:29:11.898635 (+ 1282us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:11.898647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:11.898704 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:29:11.901307 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:11.901328 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:11.901779 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:11.902159 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:11.903595 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:11.905256 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:11.906579 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:11.907344 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:11.907570 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:11.912054 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:11.912065 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:11.912079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:11.912195 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:11.915859 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:11.916605 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:11.916611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:11.916617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:11.916785 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:11.917205 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:11.917209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:11.917212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:11.917724 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:11.918117 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:11.918590 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 22:29:11.918976 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":723,"threads_started":1}
I20260806 22:29:11.929302 19315 raft_consensus.cc:348] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } }
I20260806 22:29:11.929827 19315 raft_consensus.cc:374] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:29:11.929966 19315 raft_consensus.cc:729] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c871ccb7ff8c4c4cbe57ef5eff362227, State: Initialized, Role: FOLLOWER
I20260806 22:29:11.930503 19315 consensus_queue.cc:260] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } }
I20260806 22:29:11.930841 19315 raft_consensus.cc:388] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:29:11.931018 19315 raft_consensus.cc:482] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:29:11.931216 19315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:29:11.934104 19315 raft_consensus.cc:504] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } }
I20260806 22:29:11.934772 19315 leader_election.cc:302] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c871ccb7ff8c4c4cbe57ef5eff362227; no voters:
I20260806 22:29:11.936062 19315 leader_election.cc:288] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:29:11.936506 19321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:29:11.938066 19321 raft_consensus.cc:686] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [term 1 LEADER]: Becoming Leader. State: Replica: c871ccb7ff8c4c4cbe57ef5eff362227, State: Running, Role: LEADER
I20260806 22:29:11.938741 19321 consensus_queue.cc:237] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } }
I20260806 22:29:11.939113 19315 sys_catalog.cc:564] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:29:11.943084 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c871ccb7ff8c4c4cbe57ef5eff362227. Latest consensus state: current_term: 1 leader_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } } }
I20260806 22:29:11.943094 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c871ccb7ff8c4c4cbe57ef5eff362227" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42183 } } }
I20260806 22:29:11.943955 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [sys.catalog]: This master's current role is: LEADER
I20260806 22:29:11.944048 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227 [sys.catalog]: This master's current role is: LEADER
I20260806 22:29:11.946514 19331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:29:11.956529 19331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:29:11.960798 19315 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:29:11.982028 19331 catalog_manager.cc:1347] Generated new cluster ID: 4835560fb23844469f91b290ee25cbed
I20260806 22:29:11.982236 19331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:29:12.022650 19331 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:29:12.025048 19331 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:29:12.036732 19331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: Generated new TSK 0
I20260806 22:29:12.037937 19331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:29:12.807071 19378 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:29:12.935227 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:55009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:29:13 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:29:13.551930 19388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:29:13.552407 19388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:29:13.552601 19388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:29:13.562054 19388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:29:13.562207 19388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:29:13.562366 19388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:29:13.574314 19388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.8.84:55009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:29:13.576395 19388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:29:13.578414 19388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:29:13.608737 19399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:13.613749 19401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:13.620518 19403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:29:13.621436 19388 server_base.cc:1034] running on GCE node
I20260806 22:29:13.622332 19388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:29:13.625219 19388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:29:13.626600 19388 hybrid_clock.cc:648] HybridClock initialized: now 1786055353626405 us; error 99 us; skew 500 ppm
I20260806 22:29:13.627177 19388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:29:13 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:29:13.632609 19388 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:29:13.646732 19388 webserver.cc:460] Webserver started at http://127.1.8.65:36843/ using document root <none> and password file <none>
I20260806 22:29:13.647660 19388 fs_manager.cc:359] Metadata directory not provided
I20260806 22:29:13.647956 19388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:29:13.648453 19388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:29:13 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:13.673648 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "4ac25d6005cb485caa28a52db93769d5"
format_stamp: "Formatted at 2026-08-06 22:29:13 on dist-test-slave-7vjv"
server_key: "8ecc06647f96b8aa3155fab8eadf51ac"
server_key_iv: "60e27e1037e7e71a0b787215415da373"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:13.675817 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "4ac25d6005cb485caa28a52db93769d5"
format_stamp: "Formatted at 2026-08-06 22:29:13 on dist-test-slave-7vjv"
server_key: "8ecc06647f96b8aa3155fab8eadf51ac"
server_key_iv: "60e27e1037e7e71a0b787215415da373"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:13.691545 19388 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.002s sys 0.011s
I20260806 22:29:13.725329 19412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:13.728279 19388 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.003s sys 0.015s
I20260806 22:29:13.728554 19388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "4ac25d6005cb485caa28a52db93769d5"
format_stamp: "Formatted at 2026-08-06 22:29:13 on dist-test-slave-7vjv"
server_key: "8ecc06647f96b8aa3155fab8eadf51ac"
server_key_iv: "60e27e1037e7e71a0b787215415da373"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:13.728870 19388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:29:13.755054 19388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:29:13.760807 19388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:29:13.761304 19388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:29:13.762923 19388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:29:13.766628 19388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:29:13.766767 19388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:13.766932 19388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:29:13.767011 19388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:29:13 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:13.984337 19388 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:42955
I20260806 22:29:13.984485 19582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:42955 every 8 connection(s)
I20260806 22:29:13.986400 19388 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:29:13.989108 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 19388
I20260806 22:29:13.989339 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:29:13.990509 1057 external_mini_cluster.cc:1428] Setting key a4e62c4e55bc92801b7fd092c0f57b86
Aug 06 22:29:14 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:29:14.050056 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:13.992349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:60811 (local address 127.1.8.126:42183)
0806 22:29:14.023113 (+ 30764us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:14.023119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:14.023143 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:29:14.027621 (+ 4478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:14.027631 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:14.027753 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:14.027906 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:14.031052 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:14.032997 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:14.035176 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:14.035958 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:14.036113 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:14.040930 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:14.040938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:14.040943 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:14.040993 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:14.044866 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:14.046091 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:14.046099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:14.046105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:14.046366 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:14.047065 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:14.047074 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:14.047081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:14.047636 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:14.047822 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:14.049585 (+ 1763us) server_negotiation.cc:299] Negotiation successful
0806 22:29:14.049761 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30582,"thread_start_us":30767,"threads_started":1}
I20260806 22:29:14.051430 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:13.991372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42183 (local address 127.1.8.65:60811)
0806 22:29:14.022260 (+ 30888us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:14.022337 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:14.026996 (+ 4659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:14.028592 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:14.028608 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:14.029567 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:29:14.030798 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:14.030830 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:14.033262 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:14.033266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:14.035026 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:14.035053 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:14.036253 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:14.036262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:14.036650 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:14.037470 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:29:14.037494 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:29:14.040657 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:29:14.045059 (+ 4402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:14.045071 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:14.045090 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:14.045872 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:14.046547 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:14.046554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:14.046561 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:14.046908 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:14.047909 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:29:14.047919 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:29:14.048477 (+ 558us) client_negotiation.cc:769] Sending connection context
0806 22:29:14.048952 (+ 475us) client_negotiation.cc:240] Negotiation successful
0806 22:29:14.049745 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30376,"thread_start_us":29864,"threads_started":1}
I20260806 22:29:14.053977 19583 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42183
I20260806 22:29:14.054327 19583 heartbeater.cc:464] Registering TS with master...
I20260806 22:29:14.055699 19583 heartbeater.cc:511] Master 127.1.8.126:42183 requested a full tablet report, sending...
I20260806 22:29:14.058676 19264 ts_manager.cc:194] Registered new tserver with Master: 4ac25d6005cb485caa28a52db93769d5 (127.1.8.65:42955)
I20260806 22:29:14.061425 19264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:60811
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
W20260806 22:29:14.119086 1057 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.8.66: real 0.124s user 0.000s sys 0.040s
I20260806 22:29:14.120132 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:55009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:29:14 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
W20260806 22:29:14.803510 19591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:29:14.803880 19591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:29:14.804126 19591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:29:14.812650 19591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:29:14.812865 19591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:29:14.813093 19591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:29:14.825356 19591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.8.84:55009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:29:14.826838 19591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:29:14.828433 19591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:29:14.848336 19612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:14.848317 19609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:14.850206 19608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:29:14.850781 19591 server_base.cc:1034] running on GCE node
I20260806 22:29:14.851529 19591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:29:14.852850 19591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:29:14.854095 19591 hybrid_clock.cc:648] HybridClock initialized: now 1786055354853995 us; error 93 us; skew 500 ppm
I20260806 22:29:14.854465 19591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:29:14 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055354, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:29:14.858385 19591 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:29:14.860594 19591 webserver.cc:460] Webserver started at http://127.1.8.66:41837/ using document root <none> and password file <none>
I20260806 22:29:14.861352 19591 fs_manager.cc:359] Metadata directory not provided
I20260806 22:29:14.861531 19591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:29:14.861945 19591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:29:14 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055354, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:14.879294 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "58ebd34c15b648f4a2f1e335289a165f"
format_stamp: "Formatted at 2026-08-06 22:29:14 on dist-test-slave-7vjv"
server_key: "d276e5f50f43055fee31c1294fc7aca2"
server_key_iv: "b7ad9a932c4c004ef948efdac7ff9f7d"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:14.880410 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "58ebd34c15b648f4a2f1e335289a165f"
format_stamp: "Formatted at 2026-08-06 22:29:14 on dist-test-slave-7vjv"
server_key: "d276e5f50f43055fee31c1294fc7aca2"
server_key_iv: "b7ad9a932c4c004ef948efdac7ff9f7d"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:14.886322 19591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:29:14.904954 19625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:14.906574 19591 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 22:29:14.906761 19591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "58ebd34c15b648f4a2f1e335289a165f"
format_stamp: "Formatted at 2026-08-06 22:29:14 on dist-test-slave-7vjv"
server_key: "d276e5f50f43055fee31c1294fc7aca2"
server_key_iv: "b7ad9a932c4c004ef948efdac7ff9f7d"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:14.907061 19591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:29:14.944620 19591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:29:14.949786 19591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:29:14.950263 19591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:29:14.951674 19591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:29:14.953655 19591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:29:14.953797 19591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:14.954054 19591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:29:14.954205 19591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:15.049583 19591 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:41403
I20260806 22:29:15.049682 19803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:41403 every 8 connection(s)
I20260806 22:29:15.051170 19591 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:29:15.053215 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 19591
I20260806 22:29:15.053421 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:29:15.053826 1057 external_mini_cluster.cc:1428] Setting key f85ccfdf25692f75c41beb0365ed8688
I20260806 22:29:15.065516 19583 heartbeater.cc:503] Master 127.1.8.126:42183 was elected leader, sending a full tablet report...
Aug 06 22:29:15 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055354, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:29:15.099637 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:15.056435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:47397 (local address 127.1.8.126:42183)
0806 22:29:15.073347 (+ 16912us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:15.073354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:15.073386 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:29:15.074962 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:15.074971 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:15.075109 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:15.075341 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:15.078076 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.079789 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.082928 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.083960 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.084177 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:15.092265 (+ 8088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:15.092273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:15.092277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:15.092389 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:15.095983 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:15.096830 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:15.096835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:15.096838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:15.097025 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:15.097602 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:15.097605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:15.097607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:15.097907 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:15.098082 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:15.098859 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 22:29:15.099086 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16734,"thread_start_us":16540,"threads_started":1}
I20260806 22:29:15.101075 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:15.057363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42183 (local address 127.1.8.66:47397)
0806 22:29:15.070066 (+ 12703us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:15.070123 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:15.074494 (+ 4371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:15.075701 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:15.075714 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:15.076605 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:29:15.077825 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:15.077854 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.079994 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.080000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:15.082722 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:15.082737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.084362 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.084371 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:15.084797 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:15.087279 (+ 2482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:29:15.087324 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:29:15.091839 (+ 4515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:29:15.096171 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:15.096180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:15.096199 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:15.096681 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:15.097150 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:15.097154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:15.097158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:15.097435 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:15.098082 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:29:15.098091 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:29:15.098434 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 22:29:15.098742 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 22:29:15.099337 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12287,"thread_start_us":14633,"threads_started":1}
I20260806 22:29:15.102837 19807 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42183
I20260806 22:29:15.103175 19807 heartbeater.cc:464] Registering TS with master...
I20260806 22:29:15.103924 19807 heartbeater.cc:511] Master 127.1.8.126:42183 requested a full tablet report, sending...
I20260806 22:29:15.105502 19264 ts_manager.cc:194] Registered new tserver with Master: 58ebd34c15b648f4a2f1e335289a165f (127.1.8.66:41403)
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
I20260806 22:29:15.106385 19264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:47397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:29:15.152388 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:55009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:29:15.740317 19819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:29:15.740684 19819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:29:15.740967 19819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:29:15.751163 19819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:29:15.751369 19819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:29:15.751708 19819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:29:15.764814 19819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.8.84:55009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:29:15.766264 19819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:29:15.767685 19819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:29:15.786989 19825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:15.787186 19826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:29:15.787247 19828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:29:15.788468 19819 server_base.cc:1034] running on GCE node
I20260806 22:29:15.789172 19819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:29:15.790159 19819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:29:15.791453 19819 hybrid_clock.cc:648] HybridClock initialized: now 1786055355791340 us; error 37 us; skew 500 ppm
I20260806 22:29:15.791872 19819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:29:15 dist-test-slave-7vjv krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:29:15.796478 19819 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:29:15.798945 19819 webserver.cc:460] Webserver started at http://127.1.8.67:36869/ using document root <none> and password file <none>
I20260806 22:29:15.799816 19819 fs_manager.cc:359] Metadata directory not provided
I20260806 22:29:15.799970 19819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:29:15.800388 19819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:29:15 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:15.817380 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "4d97eba8c43a4595befb2e51edc32e52"
format_stamp: "Formatted at 2026-08-06 22:29:15 on dist-test-slave-7vjv"
server_key: "da2d84f8ae123d70f8f3e6c706c1a107"
server_key_iv: "4265650db1fdeda1877a10f3b1e732ae"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:15.818347 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "4d97eba8c43a4595befb2e51edc32e52"
format_stamp: "Formatted at 2026-08-06 22:29:15 on dist-test-slave-7vjv"
server_key: "da2d84f8ae123d70f8f3e6c706c1a107"
server_key_iv: "4265650db1fdeda1877a10f3b1e732ae"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:15.825394 19819 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260806 22:29:15.844532 19840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:15.846256 19819 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 22:29:15.846519 19819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "4d97eba8c43a4595befb2e51edc32e52"
format_stamp: "Formatted at 2026-08-06 22:29:15 on dist-test-slave-7vjv"
server_key: "da2d84f8ae123d70f8f3e6c706c1a107"
server_key_iv: "4265650db1fdeda1877a10f3b1e732ae"
server_key_version: "kuduclusterkey@0"
I20260806 22:29:15.846776 19819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:29:15.871613 19819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:29:15.877218 19819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:29:15.877668 19819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:29:15.879069 19819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:29:15.881757 19819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:29:15.881929 19819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:15.882189 19819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:29:15.882319 19819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:29:15.965168 19819 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:38131
I20260806 22:29:15.965363 20002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:38131 every 8 connection(s)
I20260806 22:29:15.966748 19819 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:29:15.976646 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 19819
I20260806 22:29:15.976866 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:29:15.977353 1057 external_mini_cluster.cc:1428] Setting key f007aed28438175ad2d9cced2ceb8b2d
Aug 06 22:29:15 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:29:15.997025 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:15.971090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:38115 (local address 127.1.8.126:42183)
0806 22:29:15.972441 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:15.972450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:15.973983 (+ 1533us) server_negotiation.cc:407] Connection header received
0806 22:29:15.977763 (+ 3780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:15.977771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:15.977884 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:15.978148 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:15.980437 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.981563 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.984051 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.984769 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.984947 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:15.989218 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:15.989224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:15.989227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:15.989261 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:15.993228 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:15.994114 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:15.994119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:15.994121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:15.994243 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:15.994799 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:15.994808 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:15.994810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:15.995184 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:15.995347 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:15.996247 (+ 900us) server_negotiation.cc:299] Negotiation successful
0806 22:29:15.996507 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":837,"threads_started":1}
I20260806 22:29:15.998255 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:15.971964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42183 (local address 127.1.8.67:38115)
0806 22:29:15.973690 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:15.973762 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:15.977325 (+ 3563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:15.978429 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:15.978438 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:15.979154 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:29:15.980243 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:15.980266 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.981778 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.981782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:15.983751 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:15.983774 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:15.985245 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:15.985257 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:15.985508 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:15.986251 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:29:15.986279 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:29:15.988921 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:29:15.993402 (+ 4481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:15.993412 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:15.993438 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:15.993969 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:15.994360 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:29:15.994373 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:29:15.994377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:29:15.994667 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:29:15.995320 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:29:15.995329 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:29:15.995737 (+ 408us) client_negotiation.cc:769] Sending connection context
0806 22:29:15.996149 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 22:29:15.996785 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1404,"thread_start_us":1005,"threads_started":1}
I20260806 22:29:15.999940 20004 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42183
I20260806 22:29:16.000300 20004 heartbeater.cc:464] Registering TS with master...
I20260806 22:29:16.001233 20004 heartbeater.cc:511] Master 127.1.8.126:42183 requested a full tablet report, sending...
I20260806 22:29:16.003082 19264 ts_manager.cc:194] Registered new tserver with Master: 4d97eba8c43a4595befb2e51edc32e52 (127.1.8.67:38131)
I20260806 22:29:16.004189 19264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:38115
I20260806 22:29:16.006625 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:29:16.030418 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.013516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.1.8.126:42183)
0806 22:29:16.013680 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.013685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.014586 (+ 901us) server_negotiation.cc:407] Connection header received
0806 22:29:16.014830 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.014834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.014895 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.015006 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:16.017215 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.018177 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.019791 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.020460 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.020594 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.023531 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:16.023537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:16.023540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:16.023593 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:16.027283 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.028004 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.028017 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.028020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.028225 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.028622 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.028628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.028631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.028922 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:16.029238 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.029693 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.029930 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:29:16.035529 19264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:29:16.045843 19264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:29:16.078536 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.063457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.1.8.66:41403)
0806 22:29:16.064874 (+ 1417us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.064883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.065087 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:29:16.065490 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.065501 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.065794 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.066056 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:29:16.067891 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.070652 (+ 2761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.075005 (+ 4353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.076618 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.076773 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.077110 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.077647 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.077869 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":962,"threads_started":1}
I20260806 22:29:16.080165 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.063347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:41403 (local address 127.0.0.1:52984)
0806 22:29:16.064932 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:16.064968 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:16.065184 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:16.066525 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:16.066536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:16.066635 (+ 99us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:29:16.067654 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.067671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.070867 (+ 3196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.070876 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.074781 (+ 3905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.074797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.076903 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.076911 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.077197 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.077339 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:29:16.079110 (+ 1771us) client_negotiation.cc:240] Negotiation successful
0806 22:29:16.079360 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"thread_start_us":1027,"threads_started":1}
I20260806 22:29:16.085263 19704 tablet_service.cc:1467] Processing CreateTablet for tablet eab73d640b074b48b02d0244a272a051 (DEFAULT_TABLE table=test-table [id=2506c97762ca4d05b8160a479541aaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:29:16.087415 19704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab73d640b074b48b02d0244a272a051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:29:16.088585 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.065387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42955 (local address 127.0.0.1:57854)
0806 22:29:16.073062 (+ 7675us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:16.073087 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:16.073241 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:16.076303 (+ 3062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:16.076311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:16.076350 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:29:16.079059 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.079074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.081522 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.081527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.084628 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.084648 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.087439 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.087448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.087776 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.087836 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:29:16.087946 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:29:16.088133 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1158,"thread_start_us":934,"threads_started":1}
I20260806 22:29:16.089767 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.065680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.1.8.65:42955)
0806 22:29:16.074592 (+ 8912us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.074605 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.074643 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:29:16.074759 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.074770 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.075056 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.075349 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:29:16.079363 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.081367 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.084864 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.087057 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.087200 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.088844 (+ 1644us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.088970 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.089202 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8635,"thread_start_us":8407,"threads_started":1}
I20260806 22:29:16.092509 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.067591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.1.8.67:38131)
0806 22:29:16.080335 (+ 12744us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.080347 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.080406 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:29:16.080543 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.080552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.080791 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.081064 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:29:16.083680 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.085536 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.088876 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.090731 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.090903 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.091260 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.091687 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.091838 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12385,"spinlock_wait_cycles":18688,"thread_start_us":12340,"threads_started":1}
I20260806 22:29:16.092391 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.066848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:38131 (local address 127.0.0.1:41236)
0806 22:29:16.078294 (+ 11446us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:16.078313 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:16.078482 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:16.081030 (+ 2548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:16.081035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:16.081061 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:29:16.083389 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.083405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.085752 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.085757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.088597 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.088617 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.091187 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.091199 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.091445 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.091497 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:29:16.091607 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:29:16.091762 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11299,"thread_start_us":10932,"threads_started":1}
I20260806 22:29:16.094823 19484 tablet_service.cc:1467] Processing CreateTablet for tablet eab73d640b074b48b02d0244a272a051 (DEFAULT_TABLE table=test-table [id=2506c97762ca4d05b8160a479541aaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:29:16.096793 19484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab73d640b074b48b02d0244a272a051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:29:16.097877 19911 tablet_service.cc:1467] Processing CreateTablet for tablet eab73d640b074b48b02d0244a272a051 (DEFAULT_TABLE table=test-table [id=2506c97762ca4d05b8160a479541aaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:29:16.099848 19911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab73d640b074b48b02d0244a272a051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:29:16.108067 20027 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Bootstrap starting.
I20260806 22:29:16.109288 19807 heartbeater.cc:503] Master 127.1.8.126:42183 was elected leader, sending a full tablet report...
I20260806 22:29:16.114857 20027 tablet_bootstrap.cc:654] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Neither blocks nor log segments found. Creating new log.
I20260806 22:29:16.117408 20027 log.cc:824] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:29:16.119675 20030 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Bootstrap starting.
I20260806 22:29:16.121709 20032 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Bootstrap starting.
I20260806 22:29:16.123466 20027 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: No bootstrap required, opened a new log
I20260806 22:29:16.123847 20027 ts_tablet_manager.cc:1397] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260806 22:29:16.127641 20030 tablet_bootstrap.cc:654] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Neither blocks nor log segments found. Creating new log.
I20260806 22:29:16.128469 20032 tablet_bootstrap.cc:654] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Neither blocks nor log segments found. Creating new log.
I20260806 22:29:16.130982 20032 log.cc:824] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Log is configured to *not* fsync() on all Append() calls
I20260806 22:29:16.131217 20030 log.cc:824] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:29:16.136008 20032 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: No bootstrap required, opened a new log
I20260806 22:29:16.136452 20030 tablet_bootstrap.cc:492] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: No bootstrap required, opened a new log
I20260806 22:29:16.136559 20032 ts_tablet_manager.cc:1397] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260806 22:29:16.136853 20030 ts_tablet_manager.cc:1397] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.013s
I20260806 22:29:16.136998 20027 raft_consensus.cc:348] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.137748 20027 raft_consensus.cc:374] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:29:16.137988 20027 raft_consensus.cc:729] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ebd34c15b648f4a2f1e335289a165f, State: Initialized, Role: FOLLOWER
I20260806 22:29:16.138967 20027 consensus_queue.cc:260] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.141651 20027 ts_tablet_manager.cc:1428] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260806 22:29:16.151481 20030 raft_consensus.cc:348] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.152237 20030 raft_consensus.cc:374] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:29:16.152397 20030 raft_consensus.cc:729] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac25d6005cb485caa28a52db93769d5, State: Initialized, Role: FOLLOWER
I20260806 22:29:16.153216 20030 consensus_queue.cc:260] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.155809 20030 ts_tablet_manager.cc:1428] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Time spent starting tablet: real 0.019s user 0.008s sys 0.010s
I20260806 22:29:16.157054 20032 raft_consensus.cc:348] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.157660 20032 raft_consensus.cc:374] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:29:16.157876 20032 raft_consensus.cc:729] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d97eba8c43a4595befb2e51edc32e52, State: Initialized, Role: FOLLOWER
I20260806 22:29:16.158762 20032 consensus_queue.cc:260] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.160830 20004 heartbeater.cc:503] Master 127.1.8.126:42183 was elected leader, sending a full tablet report...
I20260806 22:29:16.161121 20032 ts_tablet_manager.cc:1428] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Time spent starting tablet: real 0.024s user 0.018s sys 0.000s
W20260806 22:29:16.220309 20005 tablet.cc:2369] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:29:16.286623 19585 tablet.cc:2369] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:29:16.304740 19809 tablet.cc:2369] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:29:16.486162 20041 raft_consensus.cc:482] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:29:16.486729 20041 raft_consensus.cc:504] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.489109 20041 leader_election.cc:288] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ac25d6005cb485caa28a52db93769d5 (127.1.8.65:42955), 4d97eba8c43a4595befb2e51edc32e52 (127.1.8.67:38131)
I20260806 22:29:16.502532 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.489660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:57733 (local address 127.1.8.65:42955)
0806 22:29:16.489973 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.489978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.491075 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 22:29:16.491534 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.491543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.491654 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.491738 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:29:16.493507 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.494830 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.499120 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.500907 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.501064 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.501196 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.502057 (+ 861us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.502189 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:29:16.502532 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.491347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:50983 (local address 127.1.8.67:38131)
0806 22:29:16.491567 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.491572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.491914 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:29:16.492182 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.492186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.492249 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.492329 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:29:16.494048 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.495306 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.499114 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.500942 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.501064 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.501215 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.502051 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.502193 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:29:16.503294 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.489749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42955 (local address 127.1.8.66:57733)
0806 22:29:16.490924 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:16.490950 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:16.491236 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:16.492080 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:16.492088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:16.492145 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:29:16.493280 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.493308 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.495004 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.495007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.498954 (+ 3947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.498966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.501558 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.501565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.501901 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.501936 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:29:16.502026 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:29:16.502181 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":175,"thread_start_us":714,"threads_started":1}
I20260806 22:29:16.503432 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.490866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:38131 (local address 127.1.8.66:50983)
0806 22:29:16.491791 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 22:29:16.491814 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:29:16.492004 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:29:16.492425 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:29:16.492430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:29:16.492493 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:29:16.493845 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.493859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.495491 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.495507 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.498894 (+ 3387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:29:16.498913 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.501457 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.501467 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:29:16.501827 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.501881 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:29:16.502007 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:29:16.502181 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":57,"thread_start_us":489,"threads_started":1}
I20260806 22:29:16.504704 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab73d640b074b48b02d0244a272a051" candidate_uuid: "58ebd34c15b648f4a2f1e335289a165f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d97eba8c43a4595befb2e51edc32e52" is_pre_election: true
I20260806 22:29:16.504637 19514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab73d640b074b48b02d0244a272a051" candidate_uuid: "58ebd34c15b648f4a2f1e335289a165f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac25d6005cb485caa28a52db93769d5" is_pre_election: true
I20260806 22:29:16.505362 19942 raft_consensus.cc:2454] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ebd34c15b648f4a2f1e335289a165f in term 0.
I20260806 22:29:16.505589 19514 raft_consensus.cc:2454] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ebd34c15b648f4a2f1e335289a165f in term 0.
I20260806 22:29:16.506098 19630 leader_election.cc:302] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d97eba8c43a4595befb2e51edc32e52, 58ebd34c15b648f4a2f1e335289a165f; no voters:
I20260806 22:29:16.506649 20041 raft_consensus.cc:2781] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:29:16.506853 20041 raft_consensus.cc:482] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:29:16.507054 20041 raft_consensus.cc:3037] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:29:16.510742 20041 raft_consensus.cc:504] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.511669 20041 leader_election.cc:288] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [CANDIDATE]: Term 1 election: Requested vote from peers 4ac25d6005cb485caa28a52db93769d5 (127.1.8.65:42955), 4d97eba8c43a4595befb2e51edc32e52 (127.1.8.67:38131)
I20260806 22:29:16.512377 19514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab73d640b074b48b02d0244a272a051" candidate_uuid: "58ebd34c15b648f4a2f1e335289a165f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac25d6005cb485caa28a52db93769d5"
I20260806 22:29:16.512372 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab73d640b074b48b02d0244a272a051" candidate_uuid: "58ebd34c15b648f4a2f1e335289a165f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d97eba8c43a4595befb2e51edc32e52"
I20260806 22:29:16.512643 19514 raft_consensus.cc:3037] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:29:16.512687 19942 raft_consensus.cc:3037] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:29:16.515692 19514 raft_consensus.cc:2454] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ebd34c15b648f4a2f1e335289a165f in term 1.
I20260806 22:29:16.515897 19942 raft_consensus.cc:2454] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ebd34c15b648f4a2f1e335289a165f in term 1.
I20260806 22:29:16.516325 19630 leader_election.cc:302] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac25d6005cb485caa28a52db93769d5, 58ebd34c15b648f4a2f1e335289a165f; no voters:
I20260806 22:29:16.516727 20041 raft_consensus.cc:2781] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:29:16.517539 20041 raft_consensus.cc:686] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 1 LEADER]: Becoming Leader. State: Replica: 58ebd34c15b648f4a2f1e335289a165f, State: Running, Role: LEADER
I20260806 22:29:16.518147 20041 consensus_queue.cc:237] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } }
I20260806 22:29:16.523524 19264 catalog_manager.cc:5495] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f reported cstate change: term changed from 0 to 1, leader changed from <none> to 58ebd34c15b648f4a2f1e335289a165f (127.1.8.66). New cstate: current_term: 1 leader_uuid: "58ebd34c15b648f4a2f1e335289a165f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ebd34c15b648f4a2f1e335289a165f" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 41403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260806 22:29:16.547672 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.541020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53000 (local address 127.1.8.66:41403)
0806 22:29:16.541253 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.541262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.541710 (+ 448us) server_negotiation.cc:407] Connection header received
0806 22:29:16.541984 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.541988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.542076 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.542232 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:29:16.542687 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.543693 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.545399 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.545862 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.546023 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.546553 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:29:16.547092 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:29:16.547255 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.547316 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.547402 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:29:16.557410 19514 raft_consensus.cc:1261] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 1 FOLLOWER]: Refusing update from remote peer 58ebd34c15b648f4a2f1e335289a165f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:29:16.557410 19942 raft_consensus.cc:1261] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 1 FOLLOWER]: Refusing update from remote peer 58ebd34c15b648f4a2f1e335289a165f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:29:16.558470 20049 consensus_queue.cc:1035] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac25d6005cb485caa28a52db93769d5" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:29:16.558854 20041 consensus_queue.cc:1035] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d97eba8c43a4595befb2e51edc32e52" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:29:16.575472 20056 mvcc.cc:205] Tried to move back new op lower bound from 7315682740447395840 to 7315682740302958592. Current Snapshot: MvccSnapshot[applied={T|T < 7315682740447395840}]
I20260806 22:29:16.580251 20058 mvcc.cc:205] Tried to move back new op lower bound from 7315682740447395840 to 7315682740302958592. Current Snapshot: MvccSnapshot[applied={T|T < 7315682740447395840}]
I20260806 22:29:16.597343 19264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:29:16.597795 19264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:29:16.602134 19264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c871ccb7ff8c4c4cbe57ef5eff362227: Sending DeleteTablet for 3 replicas of tablet eab73d640b074b48b02d0244a272a051
I20260806 22:29:16.603292 19704 tablet_service.cc:1514] Processing DeleteTablet for tablet eab73d640b074b48b02d0244a272a051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:29:16 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:52984
I20260806 22:29:16.603555 19484 tablet_service.cc:1514] Processing DeleteTablet for tablet eab73d640b074b48b02d0244a272a051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:29:16 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57854
I20260806 22:29:16.603629 19911 tablet_service.cc:1514] Processing DeleteTablet for tablet eab73d640b074b48b02d0244a272a051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:29:16 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:41236
I20260806 22:29:16.605026 20073 tablet_replica.cc:330] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: stopping tablet replica
I20260806 22:29:16.605602 20073 raft_consensus.cc:2229] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:29:16.605729 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 19388
I20260806 22:29:16.606179 20073 raft_consensus.cc:2258] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:29:16.608929 20073 ts_tablet_manager.cc:1905] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:29:16.619190 20072 tablet_replica.cc:330] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: stopping tablet replica
I20260806 22:29:16.619771 20073 ts_tablet_manager.cc:1918] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:29:16.619832 20072 raft_consensus.cc:2229] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:29:16.620110 20073 log.cc:1196] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/eab73d640b074b48b02d0244a272a051
I20260806 22:29:16.620904 20072 raft_consensus.cc:2258] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:29:16.621078 20073 ts_tablet_manager.cc:1939] T eab73d640b074b48b02d0244a272a051 P 4ac25d6005cb485caa28a52db93769d5: Deleting consensus metadata
I20260806 22:29:16.623577 19243 catalog_manager.cc:4841] TS 4ac25d6005cb485caa28a52db93769d5 (127.1.8.65:42955): tablet eab73d640b074b48b02d0244a272a051 (table test-table [id=2506c97762ca4d05b8160a479541aaeb]) successfully deleted
I20260806 22:29:16.623706 20075 tablet_replica.cc:330] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: stopping tablet replica
I20260806 22:29:16.624243 20075 raft_consensus.cc:2229] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:29:16.625173 20075 raft_consensus.cc:2258] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:29:16.625815 20072 ts_tablet_manager.cc:1905] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:29:16.627298 20075 ts_tablet_manager.cc:1905] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:29:16 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:29:16.636482 20072 ts_tablet_manager.cc:1918] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:29:16.636747 20072 log.cc:1196] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/eab73d640b074b48b02d0244a272a051
I20260806 22:29:16.637159 20075 ts_tablet_manager.cc:1918] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:29:16.637372 20075 log.cc:1196] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/eab73d640b074b48b02d0244a272a051
I20260806 22:29:16.638231 20075 ts_tablet_manager.cc:1939] T eab73d640b074b48b02d0244a272a051 P 4d97eba8c43a4595befb2e51edc32e52: Deleting consensus metadata
I20260806 22:29:16.638543 20072 ts_tablet_manager.cc:1939] T eab73d640b074b48b02d0244a272a051 P 58ebd34c15b648f4a2f1e335289a165f: Deleting consensus metadata
I20260806 22:29:16.640771 19243 catalog_manager.cc:4841] TS 4d97eba8c43a4595befb2e51edc32e52 (127.1.8.67:38131): tablet eab73d640b074b48b02d0244a272a051 (table test-table [id=2506c97762ca4d05b8160a479541aaeb]) successfully deleted
I20260806 22:29:16.641448 19243 catalog_manager.cc:4841] TS 58ebd34c15b648f4a2f1e335289a165f (127.1.8.66:41403): tablet eab73d640b074b48b02d0244a272a051 (table test-table [id=2506c97762ca4d05b8160a479541aaeb]) successfully deleted
I20260806 22:29:16.647730 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.608570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.1.8.65:42955)
0806 22:29:16.608800 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.608809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.622344 (+ 13535us) server_negotiation.cc:407] Connection header received
0806 22:29:16.622563 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.622569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.622771 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.622948 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:16.626430 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.627525 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.630082 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.630861 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.630975 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.639026 (+ 8051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:16.639075 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:16.639095 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:16.639273 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:16.644417 (+ 5144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.645243 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.645253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.645256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.645387 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.645831 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.645835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.645837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.646200 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:16.646536 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.646849 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.647117 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:29:16.648787 19578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:29:16.747673 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 19388
I20260806 22:29:16.775183 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 19591
Aug 06 22:29:16 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:29:16.793942 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.776082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53004 (local address 127.1.8.66:41403)
0806 22:29:16.776313 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.776321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.776345 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:29:16.776611 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.776621 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.776697 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.776832 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:16.778321 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.779250 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.780909 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.781384 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.781542 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.786808 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:16.786821 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:16.786836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:16.786897 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:16.790595 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.791376 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.791384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.791386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.791574 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.792032 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.792036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.792039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.792528 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:16.792933 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.793298 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.793565 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 22:29:16.794831 19797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:29:16.910276 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 19591
I20260806 22:29:16.941510 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 19819
Aug 06 22:29:16 dist-test-slave-7vjv krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:29:16.956498 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:29:16.942321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.1.8.67:38131)
0806 22:29:16.942505 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:29:16.942510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:29:16.942530 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:29:16.942692 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:29:16.942696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:29:16.942764 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:29:16.942860 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:29:16.944225 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.944974 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.946129 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:29:16.946535 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:29:16.946634 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:29:16.951045 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:29:16.951059 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:29:16.951077 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:29:16.951118 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:29:16.954047 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.954688 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.954696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.954698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.954840 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:29:16.955148 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:29:16.955151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:29:16.955153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:29:16.955510 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:29:16.955717 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 22:29:16.955992 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 22:29:16.956207 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:29:16.957244 19998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:29:17.046039 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 19819
I20260806 22:29:17.074429 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 19214
I20260806 22:29:17.075623 19309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:29:17.189680 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 19214
I20260806 22:29:17.222309 1057 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:29:17.569367 1057 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:29:17.569715 1057 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:29:17.914855 1057 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:29:17.915 UTC [18573] LOG: received smart shutdown request
2026-08-06 22:29:17.919 UTC [18573] LOG: background worker "logical replication launcher" (PID 18590) exited with exit code 1
2026-08-06 22:29:17.919 UTC [18579] LOG: shutting down
2026-08-06 22:29:18.047 UTC [18573] LOG: database system is shut down
2026-08-06T22:29:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54041 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:29:18 dist-test-slave-7vjv krb5kdc[20095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:29:18 dist-test-slave-7vjv krb5kdc[20095](info): set up 2 sockets
Aug 06 22:29:18 dist-test-slave-7vjv krb5kdc[20095](info): commencing operation
krb5kdc: starting...
W20260806 22:29:19.566639 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:29:19 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:29:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:29:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangeradmin_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangeradmin_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangerlookup_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangerlookup_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
I20260806 22:29:19.958457 1057 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/postgres -l logfile start
2026-08-06 22:29:22.455 UTC [20136] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:29:22.455 UTC [20136] LOG: listening on IPv4 address "127.1.8.84", port 53821
2026-08-06 22:29:22.458 UTC [20136] LOG: listening on Unix socket "/tmp/.s.PGSQL.53821"
2026-08-06 22:29:22.484 UTC [20139] LOG: database system was shut down at 2026-08-06 22:29:21 UTC
2026-08-06 22:29:22.488 UTC [20136] LOG: database system is ready to accept connections
I20260806 22:29:23.672364 1057 mini_postgres.cc:96] Postgres bound to 53821
2026-08-06 22:29:23.697 UTC [20152] FATAL: database "slave" does not exist
127.1.8.84:53821 - accepting connections
I20260806 22:29:23.698107 1057 mini_ranger.cc:161] Starting Ranger...
I20260806 22:29:23.729948 1057 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:29:24.230011 1057 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:29:24.230119 1057 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin
2026-08-06 22:29:24,545 [I] DB FLAVOR :POSTGRES
2026-08-06 22:29:24,546 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:29:24,546 [I] Checking connection..
2026-08-06 22:29:24,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:29:24,815 [I] Checking connection passed.
2026-08-06 22:29:24,815 [I] --------- Verifying version history table ---------
2026-08-06 22:29:24,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:29:25,070 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:29:25,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:29:25,322 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:29:25,323 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:29:25,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:29:25.574 UTC [20248] WARNING: there is no transaction in progress
2026-08-06 22:29:25,581 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:29:25,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:29:25,848 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:29:25,848 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:29:25,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:29:26,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:29:26,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-06 22:29:26,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:29:26,880 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:29:26,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:29:27,145 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:29:27,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:29:27,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:29:27,670 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:29:27,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:29:28.639 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.655 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.671 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.686 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.704 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.720 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.773 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.780 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.793 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.803 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.813 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.826 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.835 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.842 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.848 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:28.855 UTC [20412] WARNING: there is no transaction in progress
2026-08-06 22:29:29,477 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:29:29,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:29:29,732 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:29:29,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:29:29,997 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:29:29,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:29:30,256 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:29:30,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:29:30,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:29:31,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-06 22:29:31,392 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:29:31,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:29:31,628 [I] DB_PATCHES have already been applied
I20260806 22:29:31.635190 1057 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksNPSnJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:29:31.635305 1057 mini_ranger.cc:193] Using host: 127.1.8.84
I20260806 22:29:31.658061 1057 mini_ranger.cc:227] Ranger admin URL: http://127.1.8.84:34623
Aug 06, 2026 10:29:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:29:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:29:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:29:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:29:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:29:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:29:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:29:33.103312 1057 mini_ranger.cc:160] Time spent starting Ranger: real 9.405s user 0.000s sys 0.064s
Aug 06, 2026 10:29:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:29:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 10:29:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:29:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:29:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:29:53.916203 1057 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangerkms_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/rangerkms_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/HTTP_127.1.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:29:54 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055394, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:29:54.143473 1057 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:29:54.172075 1057 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:29:55.088449 1057 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:29:55.088587 1057 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms
I20260806 22:29:55.088639 1057 mini_ranger_kms.cc:227] Using postgres at 127.1.8.84:53821
2026-08-06 22:29:55,477 [I] DB FLAVOR :POSTGRES
2026-08-06 22:29:55,478 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:29:55,478 [I] Checking connection
2026-08-06 22:29:55,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:29:55,767 [I] connection success
2026-08-06 22:29:55,768 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:29:55,768 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:29:55,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:29:56,058 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:29:56,059 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:29:56,059 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:29:56,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.8.84:53821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:29:56,360 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:29:56.886461 1057 mini_ranger_kms.cc:310] Created kms service
I20260806 22:29:57.070315 1057 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:29:57.116492 1057 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:29:57 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055394, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:29:57.549357 1057 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:29:57.549683 1057 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksNPSnJ/build/asan/bin/postgresql.jar:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksNPSnJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:29:57.549727 1057 mini_ranger_kms.cc:240] Using host: 127.1.8.84
I20260806 22:29:57.577912 1057 mini_ranger_kms.cc:276] Ranger KMS PID: 20677
I20260806 22:29:57.578076 1057 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.8.84:42127
Aug 06, 2026 10:29:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:29:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:29:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:29:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:29:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:29:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:30:00 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:00.444907 1057 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.301s user 0.068s sys 0.361s
I20260806 22:30:00.445209 1057 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:30:00.445405 1057 mini_ranger_kms.cc:401] 127.1.8.84:42127/kms/v1/keys
Aug 06 22:30:00 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
Aug 06 22:30:00 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.8.84@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
Aug 06 22:30:03 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:30:03.354094 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34063
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34063
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:03.731279 20743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:03.731630 20743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:03.731792 20743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:03.739399 20743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:03.739570 20743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:03.739705 20743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:03.739830 20743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:03.739924 20743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:03.750715 20743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.8.84:42127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34063
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:03.752065 20743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:03.753468 20743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:03.765926 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:03.765977 20750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:03.765898 20752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:03.767216 20743 server_base.cc:1034] running on GCE node
I20260806 22:30:03.768020 20743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:03.769467 20743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:03.770732 20743 hybrid_clock.cc:648] HybridClock initialized: now 1786055403770679 us; error 47 us; skew 500 ppm
I20260806 22:30:03.771139 20743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:03 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:03.780267 20743 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:03.781767 20743 webserver.cc:460] Webserver started at http://127.1.8.126:34033/ using document root <none> and password file <none>
I20260806 22:30:03.782490 20743 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:03.782644 20743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:03.782991 20743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:30:03 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:03.842710 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "0948e9d16aba4f7393a1634ff87ee0f5"
format_stamp: "Formatted at 2026-08-06 22:30:03 on dist-test-slave-7vjv"
server_key: "6c19e1f00afc915f5a393fe8fb948311"
server_key_iv: "e4ff778ecf9429f717813695cbd35949"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:03.844040 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "0948e9d16aba4f7393a1634ff87ee0f5"
format_stamp: "Formatted at 2026-08-06 22:30:03 on dist-test-slave-7vjv"
server_key: "6c19e1f00afc915f5a393fe8fb948311"
server_key_iv: "e4ff778ecf9429f717813695cbd35949"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:03.856866 20743 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.009s sys 0.000s
I20260806 22:30:03.878409 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:03.880312 20743 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:30:03.880537 20743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "0948e9d16aba4f7393a1634ff87ee0f5"
format_stamp: "Formatted at 2026-08-06 22:30:03 on dist-test-slave-7vjv"
server_key: "6c19e1f00afc915f5a393fe8fb948311"
server_key_iv: "e4ff778ecf9429f717813695cbd35949"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:03.880817 20743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:03.917107 20743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:03.921041 20743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:03.921360 20743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:03.943101 20743 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34063
I20260806 22:30:03.943122 20841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34063 every 8 connection(s)
I20260806 22:30:03.944609 20743 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:03.948575 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 20743
I20260806 22:30:03.948680 20842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:03.948832 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:30:03.949522 1057 external_mini_cluster.cc:1428] Setting key 4633cbda20d6bb75701315c2d1bea93b
I20260806 22:30:03.963716 20842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Bootstrap starting.
Aug 06 22:30:03 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:03.967404 20842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:03.969115 20842 log.cc:824] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:03.972718 20842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: No bootstrap required, opened a new log
I20260806 22:30:03.975382 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:03.951640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.1.8.126:34063)
0806 22:30:03.952585 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:03.952600 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:03.952652 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:30:03.955603 (+ 2951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:03.955642 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:03.956195 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:03.956650 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:03.958004 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:03.960212 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:03.961569 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:03.962590 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:03.962785 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:03.966707 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:03.966717 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:03.966735 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:03.966789 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:03.970826 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:03.971514 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:03.971528 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:03.971536 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:03.971748 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:03.972126 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:03.972131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:03.972134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:03.972871 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:03.973205 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:03.973519 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:30:03.973995 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":401,"threads_started":1}
I20260806 22:30:03.982337 20842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:03.982700 20842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:03.982781 20842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0948e9d16aba4f7393a1634ff87ee0f5, State: Initialized, Role: FOLLOWER
I20260806 22:30:03.983492 20842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:03.983803 20842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:03.983980 20842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:03.984187 20842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:03.986572 20842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:03.987124 20842 leader_election.cc:302] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0948e9d16aba4f7393a1634ff87ee0f5; no voters:
I20260806 22:30:03.987895 20842 leader_election.cc:288] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:30:03.988072 20847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:03.989228 20847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0948e9d16aba4f7393a1634ff87ee0f5, State: Running, Role: LEADER
I20260806 22:30:03.989841 20847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:03.990574 20842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:03.992578 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } } }
I20260806 22:30:03.992940 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:03.995174 20854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:03.995106 20849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0948e9d16aba4f7393a1634ff87ee0f5. Latest consensus state: current_term: 1 leader_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } } }
I20260806 22:30:03.995549 20849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:04.001340 20854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:04.002516 20842 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:30:04.022675 20854 catalog_manager.cc:1347] Generated new cluster ID: c525a26f0e3a4a859eaad532c6885803
I20260806 22:30:04.022914 20854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:04.038627 20854 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:30:04.040864 20854 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:04.050578 20854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Generated new TSK 0
I20260806 22:30:04.051766 20854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:30:04.869294 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:05 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:30:05.294816 20909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:05.295252 20909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:05.295467 20909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:05.305807 20909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:05.305927 20909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:05.306094 20909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:05.320645 20909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:05.322258 20909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:05.323813 20909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:05.352761 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:05.352839 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:05.365459 20925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:05.366238 20909 server_base.cc:1034] running on GCE node
I20260806 22:30:05.367274 20909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:05.372663 20909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:05.374010 20909 hybrid_clock.cc:648] HybridClock initialized: now 1786055405373925 us; error 79 us; skew 500 ppm
I20260806 22:30:05.374418 20909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:05 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:05.378228 20909 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:05.381752 20909 webserver.cc:460] Webserver started at http://127.1.8.65:46303/ using document root <none> and password file <none>
I20260806 22:30:05.382589 20909 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:05.382768 20909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:05.383152 20909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:30:05 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:05.413465 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "b8777ff4f350488da8362bcf425dbd28"
format_stamp: "Formatted at 2026-08-06 22:30:05 on dist-test-slave-7vjv"
server_key: "2926f1889a59f92c2e636be2fd579086"
server_key_iv: "f31b29ef394384ac4d5d03e4b0018615"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:05.414891 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "b8777ff4f350488da8362bcf425dbd28"
format_stamp: "Formatted at 2026-08-06 22:30:05 on dist-test-slave-7vjv"
server_key: "2926f1889a59f92c2e636be2fd579086"
server_key_iv: "f31b29ef394384ac4d5d03e4b0018615"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:05.427964 20909 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.003s sys 0.007s
I20260806 22:30:05.456840 20933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:05.458702 20909 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.008s
I20260806 22:30:05.458925 20909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "b8777ff4f350488da8362bcf425dbd28"
format_stamp: "Formatted at 2026-08-06 22:30:05 on dist-test-slave-7vjv"
server_key: "2926f1889a59f92c2e636be2fd579086"
server_key_iv: "f31b29ef394384ac4d5d03e4b0018615"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:05.459193 20909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:05.482736 20909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:05.488492 20909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:05.488940 20909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:05.496599 20909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:05.509563 20909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:05.509781 20909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:05.510021 20909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:05.510152 20909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:05.673559 20909 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:33041
I20260806 22:30:05.673591 21105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:33041 every 8 connection(s)
I20260806 22:30:05.675562 20909 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:05.681648 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 20909
I20260806 22:30:05.681860 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:30:05.682693 1057 external_mini_cluster.cc:1428] Setting key 030cdba2b073d306044941c8d77dbaac
Aug 06 22:30:05 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
Aug 06 22:30:05 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:05.727651 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:05.679838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:56457 (local address 127.1.8.126:34063)
0806 22:30:05.685123 (+ 5285us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:05.685132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:05.692152 (+ 7020us) server_negotiation.cc:407] Connection header received
0806 22:30:05.697637 (+ 5485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:05.697644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:05.697751 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:05.697834 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:05.701376 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:05.702633 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:05.712349 (+ 9716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:05.713121 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:05.713303 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:05.718193 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:05.718201 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:05.718206 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:05.718252 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:05.722187 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:05.723320 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:05.723326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:05.723330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:05.723505 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:05.724153 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:05.724158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:05.724162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:05.724531 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:05.724665 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:05.726946 (+ 2281us) server_negotiation.cc:299] Negotiation successful
0806 22:30:05.727202 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5089,"thread_start_us":4917,"threads_started":1}
I20260806 22:30:05.729743 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:05.679215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.65:56457)
0806 22:30:05.685781 (+ 6566us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:05.685833 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:05.697187 (+ 11354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:05.698519 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:05.698533 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:05.700176 (+ 1643us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:05.701135 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:05.701164 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:05.702880 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:05.702886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:05.705629 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:05.705646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:05.713453 (+ 7807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:05.713462 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:05.713816 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:05.714740 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:05.714774 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:05.717880 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:05.722392 (+ 4512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:05.722403 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:05.722425 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:05.723161 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:05.723659 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:05.723664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:05.723668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:05.723987 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:05.724769 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:05.724778 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:05.725236 (+ 458us) client_negotiation.cc:769] Sending connection context
0806 22:30:05.727981 (+ 2745us) client_negotiation.cc:240] Negotiation successful
0806 22:30:05.728656 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6246,"thread_start_us":6012,"threads_started":1}
I20260806 22:30:05.731642 21107 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:05.731994 21107 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:05.732996 21107 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:05.735888 20789 ts_manager.cc:194] Registered new tserver with Master: b8777ff4f350488da8362bcf425dbd28 (127.1.8.65:33041)
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
I20260806 22:30:05.738127 20789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:56457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
W20260806 22:30:05.794112 1057 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.8.66: real 0.107s user 0.000s sys 0.028s
I20260806 22:30:05.795100 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:06 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
W20260806 22:30:06.280328 21116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:06.280889 21116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:06.281116 21116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:06.292863 21116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:06.293118 21116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:06.293365 21116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:06.309542 21116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:06.311749 21116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:06.313781 21116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:06.355165 21127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:06.364487 21133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:06.365605 21128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:06.372542 21116 server_base.cc:1034] running on GCE node
I20260806 22:30:06.373661 21116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:06.380450 21116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:06.388055 21116 hybrid_clock.cc:648] HybridClock initialized: now 1786055406387941 us; error 113 us; skew 500 ppm
I20260806 22:30:06.388581 21116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:06 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:06.393250 21116 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:06.397661 21116 webserver.cc:460] Webserver started at http://127.1.8.66:33803/ using document root <none> and password file <none>
I20260806 22:30:06.398703 21116 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:06.398873 21116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:06.399262 21116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:30:06 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:06.419477 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "13dc439193424ba1bc25da05d6a35683"
format_stamp: "Formatted at 2026-08-06 22:30:06 on dist-test-slave-7vjv"
server_key: "234d9104d49672048b615465dcced9ae"
server_key_iv: "82ceee1110affe3613602dc6a4e36c9d"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:06.421957 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "13dc439193424ba1bc25da05d6a35683"
format_stamp: "Formatted at 2026-08-06 22:30:06 on dist-test-slave-7vjv"
server_key: "234d9104d49672048b615465dcced9ae"
server_key_iv: "82ceee1110affe3613602dc6a4e36c9d"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:06.430150 21116 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260806 22:30:06.449678 21155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:06.451632 21116 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 22:30:06.451831 21116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "13dc439193424ba1bc25da05d6a35683"
format_stamp: "Formatted at 2026-08-06 22:30:06 on dist-test-slave-7vjv"
server_key: "234d9104d49672048b615465dcced9ae"
server_key_iv: "82ceee1110affe3613602dc6a4e36c9d"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:06.452142 21116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:06.477174 21116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:06.481537 21116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:06.481873 21116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:06.482900 21116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:06.484562 21116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:06.484640 21116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:06.484776 21116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:06.484827 21116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:06.523999 21116 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40755
I20260806 22:30:06.524034 21332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40755 every 8 connection(s)
I20260806 22:30:06.525481 21116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:06.526242 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 21116
I20260806 22:30:06.526414 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:30:06.527029 1057 external_mini_cluster.cc:1428] Setting key 0967bb2efebc582ea14b7e4ff6e4f384
Aug 06 22:30:06 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
I20260806 22:30:06.567579 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:06.530885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:32769 (local address 127.1.8.126:34063)
0806 22:30:06.548665 (+ 17780us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:06.548675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:06.551593 (+ 2918us) server_negotiation.cc:407] Connection header received
0806 22:30:06.551658 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:06.551661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:06.551745 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:06.551879 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:06.553605 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:06.554357 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:06.555972 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:06.556443 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:06.556584 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:06.560612 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:06.560617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:06.560620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:06.560663 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:06.563775 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:06.564634 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:06.564639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:06.564642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:06.564894 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:06.565435 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:06.565441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:06.565444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:06.565825 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:06.566010 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:06.566901 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 22:30:06.567147 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17581,"thread_start_us":17446,"threads_started":1}
I20260806 22:30:06.568928 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:06.530073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.66:32769)
0806 22:30:06.549387 (+ 19314us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:06.549444 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:06.551274 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:06.552208 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:06.552218 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:06.552697 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:06.553398 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:06.553426 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:06.554532 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:06.554539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:06.555803 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:06.555816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:06.556706 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:06.556714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:06.557007 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:06.557855 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:06.557906 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:06.560357 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:06.563925 (+ 3568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:06.563935 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:06.563952 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:06.564497 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:06.565052 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:06.565058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:06.565062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:06.565296 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:06.565990 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:06.565999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:06.566472 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 22:30:06.566787 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 22:30:06.567440 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18964,"thread_start_us":18698,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
I20260806 22:30:06.570731 21334 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:06.571161 21334 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:06.572045 21334 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:06.574012 20789 ts_manager.cc:194] Registered new tserver with Master: 13dc439193424ba1bc25da05d6a35683 (127.1.8.66:40755)
I20260806 22:30:06.575285 20789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:32769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:30:06.615608 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:06.742192 21107 heartbeater.cc:503] Master 127.1.8.126:34063 was elected leader, sending a full tablet report...
W20260806 22:30:07.011890 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:07.012212 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:07.012319 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:07.019569 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:07.019742 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:07.019949 21340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:07.029917 21340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:07.031148 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:07.032615 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:07.046527 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:07.046535 21352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:07.046442 21349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:07.047730 21340 server_base.cc:1034] running on GCE node
I20260806 22:30:07.048420 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:07.049384 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:07.050611 21340 hybrid_clock.cc:648] HybridClock initialized: now 1786055407050567 us; error 59 us; skew 500 ppm
I20260806 22:30:07.050932 21340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:07.054154 21340 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:07.055701 21340 webserver.cc:460] Webserver started at http://127.1.8.67:40245/ using document root <none> and password file <none>
I20260806 22:30:07.056468 21340 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:07.056622 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:07.056936 21340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:07.074010 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "5be4d7598cec4c0fa22c2b7795600d47"
format_stamp: "Formatted at 2026-08-06 22:30:07 on dist-test-slave-7vjv"
server_key: "41bb409979b9bc5bf13a44c222d9edf1"
server_key_iv: "0c1547fed84d037a1330cab7927e4d78"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:07.074875 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "5be4d7598cec4c0fa22c2b7795600d47"
format_stamp: "Formatted at 2026-08-06 22:30:07 on dist-test-slave-7vjv"
server_key: "41bb409979b9bc5bf13a44c222d9edf1"
server_key_iv: "0c1547fed84d037a1330cab7927e4d78"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:07.080472 21340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 22:30:07.096801 21363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:07.098320 21340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:30:07.098569 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5be4d7598cec4c0fa22c2b7795600d47"
format_stamp: "Formatted at 2026-08-06 22:30:07 on dist-test-slave-7vjv"
server_key: "41bb409979b9bc5bf13a44c222d9edf1"
server_key_iv: "0c1547fed84d037a1330cab7927e4d78"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:07.098805 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:07.114441 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:07.118577 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:07.118916 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:07.119974 21340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:07.121564 21340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:07.121666 21340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:07.121833 21340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:07.121896 21340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:07.161655 21340 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:36451
I20260806 22:30:07.161741 21529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:36451 every 8 connection(s)
I20260806 22:30:07.163053 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:07.172582 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 21340
I20260806 22:30:07.172757 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:30:07.173243 1057 external_mini_cluster.cc:1428] Setting key 6b916ab353939671db106ee808f3c7db
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:07.184463 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:07.166077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:43717 (local address 127.1.8.126:34063)
0806 22:30:07.166697 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:07.166703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:07.167723 (+ 1020us) server_negotiation.cc:407] Connection header received
0806 22:30:07.169707 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:07.169712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:07.169804 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:07.169940 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:07.171427 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.172388 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.174131 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.174606 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.174756 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:07.178376 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:07.178382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:07.178385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:07.178416 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:07.181301 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.182063 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.182070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.182073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.182220 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.182667 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.182673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.182676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.182959 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:07.183117 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:07.183795 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 22:30:07.184038 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":304,"threads_started":1}
I20260806 22:30:07.185412 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:07.166699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.67:43717)
0806 22:30:07.167556 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:07.167608 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:07.169444 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:07.170202 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:07.170209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:07.170640 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:07.171255 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:07.171272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.172562 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.172567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:07.173983 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:07.174000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.174869 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.174878 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:07.175151 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:07.175882 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:07.175904 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:07.178160 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:07.181447 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:07.181456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:07.181471 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:07.181929 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:07.182335 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:07.182340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:07.182344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:07.182551 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:07.183089 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:07.183096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:07.183429 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 22:30:07.183695 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 22:30:07.184251 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":406,"threads_started":1}
I20260806 22:30:07.187126 21530 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:07.187466 21530 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:07.188567 21530 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:07.190179 20789 ts_manager.cc:194] Registered new tserver with Master: 5be4d7598cec4c0fa22c2b7795600d47 (127.1.8.67:36451)
I20260806 22:30:07.191015 20789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:43717
I20260806 22:30:07.201239 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:30:07.203125 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 20909
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:30:07.220832 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:07.204031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.1.8.65:33041)
0806 22:30:07.204789 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:07.204796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:07.204826 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:30:07.204883 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:07.204889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:07.205108 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:07.205252 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:07.206784 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.207835 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.209411 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.209985 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.210086 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:07.214715 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:07.214729 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:07.214744 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:07.214778 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:07.217899 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.218648 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.218655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.218658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.218823 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.219197 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.219200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.219202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.219560 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:07.219844 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:07.220147 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:30:07.220385 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":398,"threads_started":1}
I20260806 22:30:07.222239 21098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:07.307253 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 20909
I20260806 22:30:07.332912 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 21116
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:30:07.355093 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:07.338710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.1.8.66:40755)
0806 22:30:07.339378 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:07.339385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:07.339413 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:30:07.339483 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:07.339490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:07.339739 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:07.339873 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:07.341333 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.342441 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.343726 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.344383 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.344490 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:07.349356 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:07.349368 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:07.349383 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:07.349418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:07.352362 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.353052 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.353059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.353061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.353204 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.353548 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.353551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.353553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.353905 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:07.354146 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:07.354384 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:30:07.354594 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":379,"threads_started":1}
I20260806 22:30:07.356454 21327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:07.438782 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 21116
I20260806 22:30:07.463745 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 21340
Aug 06 22:30:07 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:30:07.480834 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:07.464891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.1.8.67:36451)
0806 22:30:07.465533 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:07.465540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:07.465570 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:30:07.465631 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:07.465637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:07.465860 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:07.465992 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:07.467253 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.468379 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.469727 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:07.470324 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:07.470455 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:07.474851 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:07.474863 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:07.474880 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:07.474928 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:07.478045 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.478720 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.478727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.478730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.478873 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:07.479208 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:07.479212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:07.479214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:07.479584 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:07.479837 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:07.480079 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:30:07.480436 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":329,"threads_started":1}
I20260806 22:30:07.482251 21521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:07.566416 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 21340
I20260806 22:30:07.593844 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 20743
I20260806 22:30:07.595270 20830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:07.705145 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 20743
I20260806 22:30:07.748363 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34063
--webserver_interface=127.1.8.126
--webserver_port=34033
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34063
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:08.105597 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:08.105975 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:08.106142 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:08.113502 21549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:08.113644 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:08.113755 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:08.113852 21549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:08.113951 21549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:08.124459 21549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34063
--ranger_config_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=34033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:08.125826 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:08.127259 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:08.139896 21561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:08.139988 21560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:08.139961 21564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:08.145625 21549 server_base.cc:1034] running on GCE node
I20260806 22:30:08.146499 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:08.148015 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:08.149288 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786055408149241 us; error 58 us; skew 500 ppm
I20260806 22:30:08.149657 21549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:08 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:08.152848 21549 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:08.154469 21549 webserver.cc:460] Webserver started at http://127.1.8.126:34033/ using document root <none> and password file <none>
I20260806 22:30:08.155172 21549 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:08.155325 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:30:08 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:08.215304 21549 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:30:08.218562 21574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:08.220202 21549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:30:08.220402 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "0948e9d16aba4f7393a1634ff87ee0f5"
format_stamp: "Formatted at 2026-08-06 22:30:03 on dist-test-slave-7vjv"
server_key: "6c19e1f00afc915f5a393fe8fb948311"
server_key_iv: "e4ff778ecf9429f717813695cbd35949"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:08.221441 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:08.240926 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:08.244678 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:08.244984 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:08.264111 21549 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34063
I20260806 22:30:08.264185 21655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34063 every 8 connection(s)
I20260806 22:30:08.265753 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:08.270660 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 21549
I20260806 22:30:08.271569 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:33041
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:34063
--webserver_port=46303
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:08.273067 21656 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:30:08.276494 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Bootstrap starting.
I20260806 22:30:08.299503 21656 log.cc:824] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:08.311575 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:30:08.312251 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Bootstrap complete.
I20260806 22:30:08.326107 21656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:08.327299 21656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0948e9d16aba4f7393a1634ff87ee0f5, State: Initialized, Role: FOLLOWER
I20260806 22:30:08.328019 21656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:08.328317 21656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:08.328439 21656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:08.328631 21656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:30:08.334281 21656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:08.334888 21656 leader_election.cc:302] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0948e9d16aba4f7393a1634ff87ee0f5; no voters:
I20260806 22:30:08.335618 21656 leader_election.cc:288] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:30:08.335791 21660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:30:08.341941 21656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:08.343093 21660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [term 2 LEADER]: Becoming Leader. State: Replica: 0948e9d16aba4f7393a1634ff87ee0f5, State: Running, Role: LEADER
I20260806 22:30:08.343961 21660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } }
I20260806 22:30:08.346953 21661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } } }
I20260806 22:30:08.347318 21661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:08.349503 21671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:08.352025 21662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0948e9d16aba4f7393a1634ff87ee0f5. Latest consensus state: current_term: 2 leader_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0948e9d16aba4f7393a1634ff87ee0f5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34063 } } }
I20260806 22:30:08.356225 21662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:08.356127 21671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:08.357882 21671 catalog_manager.cc:1259] Loaded cluster ID: c525a26f0e3a4a859eaad532c6885803
I20260806 22:30:08.358036 21671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:08.362347 21671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:08.363806 21671 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Loaded TSK: 0
I20260806 22:30:08.365429 21671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:30:08.371873 21656 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 22:30:08.769330 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:08.769814 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:08.770044 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:08.781908 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:08.782168 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:08.782433 21658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:08.798877 21658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:33041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=46303
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:08.801074 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:08.803025 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:08.822587 21723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:08.822582 21720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:08.823486 21725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:08.828534 21658 server_base.cc:1034] running on GCE node
I20260806 22:30:08.829416 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:08.830801 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:08.832110 21658 hybrid_clock.cc:648] HybridClock initialized: now 1786055408832029 us; error 40 us; skew 500 ppm
I20260806 22:30:08.832513 21658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:08 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:08.839382 21658 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:08.841114 21658 webserver.cc:460] Webserver started at http://127.1.8.65:46303/ using document root <none> and password file <none>
I20260806 22:30:08.841861 21658 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:08.842130 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:30:08 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:08.881367 21658 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.007s sys 0.004s
I20260806 22:30:08.888693 21734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:08.890280 21658 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.008s sys 0.000s
I20260806 22:30:08.890496 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "b8777ff4f350488da8362bcf425dbd28"
format_stamp: "Formatted at 2026-08-06 22:30:05 on dist-test-slave-7vjv"
server_key: "2926f1889a59f92c2e636be2fd579086"
server_key_iv: "f31b29ef394384ac4d5d03e4b0018615"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:08.891691 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:08.917330 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:08.922943 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:08.923316 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:08.930752 21658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:08.938086 21658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:08.938279 21658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 22:30:08.938421 21658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:08.938503 21658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:09.008847 21658 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:33041
I20260806 22:30:09.008940 21918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:33041 every 8 connection(s)
I20260806 22:30:09.010804 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:09.012180 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 21658
I20260806 22:30:09.013010 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:40755
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:34063
--webserver_port=33803
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:09 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:09.065986 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:09.018664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.65:55455)
0806 22:30:09.033559 (+ 14895us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:09.033622 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:09.038603 (+ 4981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:09.041146 (+ 2543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:09.041156 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:09.041836 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:09.042672 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:09.042700 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.045539 (+ 2839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.045545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:09.047638 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:09.047654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.049768 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.049779 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:09.050106 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:09.051016 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:09.051063 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:09.054011 (+ 2948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:09.059042 (+ 5031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:09.059053 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:09.059071 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:09.059955 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:09.060631 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:09.060636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:09.060641 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:09.060927 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:09.063198 (+ 2271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:09.063206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:09.063618 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 22:30:09.063933 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 22:30:09.064550 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14545,"thread_start_us":15017,"threads_started":1}
I20260806 22:30:09.067040 21924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:09.019981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:55455 (local address 127.1.8.126:34063)
0806 22:30:09.021480 (+ 1499us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:09.021505 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:09.033801 (+ 12296us) server_negotiation.cc:407] Connection header received
0806 22:30:09.039368 (+ 5567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:09.039390 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:09.039911 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:09.040329 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:09.042923 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.045256 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.047824 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.049087 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.049303 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:09.054358 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:09.054370 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:09.054404 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:09.054472 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:09.058838 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:09.060194 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:09.060204 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:09.060213 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:09.060472 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:09.062176 (+ 1704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:09.062182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:09.062186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:09.063014 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:09.063250 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:09.065166 (+ 1916us) server_negotiation.cc:299] Negotiation successful
0806 22:30:09.065654 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1194,"thread_start_us":303,"threads_started":1}
I20260806 22:30:09.069980 21919 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:09.070302 21919 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:09.071066 21919 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
W20260806 22:30:09.073933 21919 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:34063 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:30:09.074167 21919 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:09.074417 21919 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:09.075595 21919 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:09.075862 21919 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
W20260806 22:30:09.076915 21919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:30:09.463250 21925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:09.463562 21925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:09.463757 21925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:09.470904 21925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:09.471050 21925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:09.471251 21925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:09.480978 21925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:40755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=33803
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:09.482260 21925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:09.483632 21925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:09.498167 21943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:09.498740 21947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:09.499295 21942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:09.500118 21925 server_base.cc:1034] running on GCE node
I20260806 22:30:09.500815 21925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:09.501989 21925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:09.503235 21925 hybrid_clock.cc:648] HybridClock initialized: now 1786055409503128 us; error 88 us; skew 500 ppm
I20260806 22:30:09.503613 21925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:09 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:09.507676 21925 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:09.534667 21925 webserver.cc:460] Webserver started at http://127.1.8.66:33803/ using document root <none> and password file <none>
I20260806 22:30:09.535841 21925 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:09.537613 21925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:30:09 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:09.564357 21925 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:30:09.573459 21959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:09.575574 21925 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.006s
I20260806 22:30:09.575865 21925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "13dc439193424ba1bc25da05d6a35683"
format_stamp: "Formatted at 2026-08-06 22:30:06 on dist-test-slave-7vjv"
server_key: "234d9104d49672048b615465dcced9ae"
server_key_iv: "82ceee1110affe3613602dc6a4e36c9d"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:09.577451 21925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:09.613348 21925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Aug 06 22:30:09 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:09.619150 21925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:09.619532 21925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:09.620889 21925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:09.622815 21925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:09.622977 21925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:09.623122 21925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:09.623230 21925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:09.703476 21925 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40755
I20260806 22:30:09.704444 22140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40755 every 8 connection(s)
I20260806 22:30:09.705329 21925 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:09.711536 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 21925
I20260806 22:30:09.712332 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.8.84:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:36451
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:34063
--webserver_port=40245
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:09 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:09.744887 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:09.708584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:36929 (local address 127.1.8.126:34063)
0806 22:30:09.720095 (+ 11511us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:09.720102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:09.724637 (+ 4535us) server_negotiation.cc:407] Connection header received
0806 22:30:09.724715 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:09.724721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:09.724813 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:09.724937 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:09.727318 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.728511 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.731014 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.731702 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.731805 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:09.736833 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:09.736840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:09.736845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:09.736895 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:09.740682 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:09.741702 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:09.741708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:09.741711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:09.741912 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:09.742582 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:09.742587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:09.742591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:09.742962 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:09.743098 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:09.744272 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0806 22:30:09.744516 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11362,"thread_start_us":14625,"threads_started":1}
I20260806 22:30:09.747673 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:09.709325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.66:36929)
0806 22:30:09.720684 (+ 11359us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:09.720741 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:09.723296 (+ 2555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:09.725414 (+ 2118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:09.725427 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:09.726162 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:09.727089 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:09.727116 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.728751 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.728757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:09.730838 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:09.730853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:09.732224 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:09.732235 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:09.732558 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:09.733389 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:09.733434 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:09.736561 (+ 3127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:09.740862 (+ 4301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:09.740871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:09.740891 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:09.741549 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:09.742090 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:09.742095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:09.742099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:09.742445 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:09.743234 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:09.743244 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:09.743768 (+ 524us) client_negotiation.cc:769] Sending connection context
0806 22:30:09.745507 (+ 1739us) client_negotiation.cc:240] Negotiation successful
0806 22:30:09.746126 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11002,"thread_start_us":14267,"threads_started":1}
I20260806 22:30:09.749881 22141 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:09.750223 22141 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:09.751137 22141 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:09.753443 21601 ts_manager.cc:194] Registered new tserver with Master: 13dc439193424ba1bc25da05d6a35683 (127.1.8.66:40755)
I20260806 22:30:09.755546 21601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:36929
Aug 06 22:30:10 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:10.077080 21919 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:10.077610 21919 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:10.078958 21601 ts_manager.cc:194] Registered new tserver with Master: b8777ff4f350488da8362bcf425dbd28 (127.1.8.65:33041)
I20260806 22:30:10.080169 21601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:55455
W20260806 22:30:10.178040 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:10.178411 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:10.178644 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:10.189935 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:10.190083 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:10.190307 22147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:10.205715 22147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.8.84:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:36451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=40245
--tserver_master_addrs=127.1.8.126:34063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:10.207595 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:10.209338 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:10.226487 22157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:10.228528 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:10.241842 22165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:10.242069 22147 server_base.cc:1034] running on GCE node
I20260806 22:30:10.242905 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:10.244174 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:10.245434 22147 hybrid_clock.cc:648] HybridClock initialized: now 1786055410245373 us; error 62 us; skew 500 ppm
I20260806 22:30:10.245776 22147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:10 dist-test-slave-7vjv krb5kdc[20095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:10.249145 22147 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:10.252431 22147 webserver.cc:460] Webserver started at http://127.1.8.67:40245/ using document root <none> and password file <none>
I20260806 22:30:10.253340 22147 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:10.253518 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:30:10 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:10.285879 22147 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.004s
I20260806 22:30:10.289695 22180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:10.291524 22147 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:30:10.291718 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5be4d7598cec4c0fa22c2b7795600d47"
format_stamp: "Formatted at 2026-08-06 22:30:07 on dist-test-slave-7vjv"
server_key: "41bb409979b9bc5bf13a44c222d9edf1"
server_key_iv: "0c1547fed84d037a1330cab7927e4d78"
server_key_version: "kuduclusterkey@0"
I20260806 22:30:10.293143 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:10.329972 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:10.335557 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:10.336007 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:10.342739 22147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:10.344817 22147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:10.344997 22147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 22:30:10.345139 22147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:10.345232 22147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:30:10 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for HTTP/127.1.8.84@KRBTEST.COM
I20260806 22:30:10.633531 22147 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:36451
I20260806 22:30:10.633574 22351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:36451 every 8 connection(s)
I20260806 22:30:10.635897 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:10.643414 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 22147
Aug 06 22:30:10 dist-test-slave-7vjv krb5kdc[20095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:10.700141 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.644441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.1.8.126:34063)
0806 22:30:10.680764 (+ 36323us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.680770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.680789 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:30:10.680850 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.680856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.680939 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.681081 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:10.683498 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.684634 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.686692 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.687366 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.687514 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.691660 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:10.691667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:10.691672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:10.691711 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:10.695359 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.696347 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.696353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.696356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.696518 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.696991 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.696995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.696998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.697390 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:10.697662 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.699393 (+ 1731us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.699681 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36197,"thread_start_us":35763,"threads_started":1}
I20260806 22:30:10.704854 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.641808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:38753 (local address 127.1.8.126:34063)
0806 22:30:10.676556 (+ 34748us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.676563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.680342 (+ 3779us) server_negotiation.cc:407] Connection header received
0806 22:30:10.681721 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.681726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.681813 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.681892 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:10.684424 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.685768 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.688446 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.689227 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.689335 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.697446 (+ 8111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:10.697455 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:10.697461 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:10.697521 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:10.701301 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.702359 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.702375 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.702378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.702519 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.703094 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.703099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.703103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.703359 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:10.703465 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.704257 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.704461 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34582,"thread_start_us":39381,"threads_started":1}
I20260806 22:30:10.705891 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.642462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34063 (local address 127.1.8.67:38753)
0806 22:30:10.677375 (+ 34913us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.677449 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.681317 (+ 3868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.682544 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.682556 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.683306 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:10.684197 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.684221 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.685998 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.686003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.688255 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.688272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.689747 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.689755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.690100 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.690876 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:10.690912 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:10.697109 (+ 6197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:10.701518 (+ 4409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:10.701528 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:10.701547 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:10.702216 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:10.702677 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:10.702681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:10.702685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:10.702964 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:10.703542 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:10.703548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:10.703856 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.704130 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.704705 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34548,"thread_start_us":37517,"threads_started":1}
I20260806 22:30:10.707347 22352 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34063
I20260806 22:30:10.707688 22352 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:10.708628 22352 heartbeater.cc:511] Master 127.1.8.126:34063 requested a full tablet report, sending...
I20260806 22:30:10.710371 21601 ts_manager.cc:194] Registered new tserver with Master: 5be4d7598cec4c0fa22c2b7795600d47 (127.1.8.67:36451)
I20260806 22:30:10.711485 21601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:38753
I20260806 22:30:10.720186 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:30:10.739531 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.724261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.1.8.126:34063)
0806 22:30:10.724527 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.724537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.725015 (+ 478us) server_negotiation.cc:407] Connection header received
0806 22:30:10.725229 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.725235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.725311 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.725392 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:10.727530 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.728397 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.730282 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.730678 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.730766 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.734056 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:10.734061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:10.734064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:10.734093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:10.736789 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.737589 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.737593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.737595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.737711 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:10.738163 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:10.738166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:10.738168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:10.738431 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:10.738607 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.739015 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.739184 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:30:10.745316 21601 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:30:10.752970 21601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:30:10.760891 22141 heartbeater.cc:503] Master 127.1.8.126:34063 was elected leader, sending a full tablet report...
I20260806 22:30:10.803650 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.768327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:33041 (local address 127.0.0.1:46482)
0806 22:30:10.792711 (+ 24384us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.792768 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.793105 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.794743 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.794752 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.794787 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:10.795244 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.795258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.797431 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.797437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.800225 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.800240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.802434 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.802440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.802745 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.802810 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.802952 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.803174 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24107,"mutex_wait_us":4,"spinlock_wait_cycles":7424,"thread_start_us":26115,"threads_started":1}
I20260806 22:30:10.804919 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.768647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.1.8.65:33041)
0806 22:30:10.793814 (+ 25167us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.793822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.793851 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:30:10.793952 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.793961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.794241 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.794449 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:10.795478 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.797218 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.800413 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.801881 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.802023 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.802265 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.804389 (+ 2124us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.804602 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24961,"thread_start_us":25466,"threads_started":1}
I20260806 22:30:10.808828 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.770838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.1.8.66:40755)
0806 22:30:10.791677 (+ 20839us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.791691 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.794856 (+ 3165us) server_negotiation.cc:407] Connection header received
0806 22:30:10.795173 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.795181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.795488 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.795693 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:10.797937 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.799575 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.805082 (+ 5507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.806599 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.806697 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.806943 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.808286 (+ 1343us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.808460 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20559,"thread_start_us":20854,"threads_started":1}
I20260806 22:30:10.809819 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.770187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40755 (local address 127.0.0.1:54308)
0806 22:30:10.794743 (+ 24556us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.794760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.794978 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.795981 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.795987 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.796012 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:10.796364 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.796377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.799793 (+ 3416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.799798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.804137 (+ 4339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.804154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.807857 (+ 3703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.807864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.808064 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.808097 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.809271 (+ 1174us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.809417 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24430,"mutex_wait_us":978,"spinlock_wait_cycles":6272,"thread_start_us":22088,"threads_started":1}
I20260806 22:30:10.809522 21805 tablet_service.cc:1467] Processing CreateTablet for tablet 309776c3bc864cfe84bad4160ea69449 (DEFAULT_TABLE table=test-table [id=5b037aa46f0a40c788f416b907316e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:10.811305 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309776c3bc864cfe84bad4160ea69449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:10.811961 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.801230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:36451 (local address 127.0.0.1:60536)
0806 22:30:10.802967 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.802985 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.803088 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.804566 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.804570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.804594 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:10.804825 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.804832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.806724 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.806728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.808924 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.808941 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.811184 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.811192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.811382 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.811415 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.811507 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.811638 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1597,"thread_start_us":1436,"threads_started":1}
I20260806 22:30:10.812790 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.795190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.1.8.67:36451)
0806 22:30:10.803580 (+ 8390us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.803591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.803620 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:30:10.803730 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.803740 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.803978 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.804223 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:10.805035 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.806549 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.809139 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.810622 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.810754 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.811018 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.812300 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.812460 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8205,"thread_start_us":8088,"threads_started":1}
I20260806 22:30:10.813964 22031 tablet_service.cc:1467] Processing CreateTablet for tablet 309776c3bc864cfe84bad4160ea69449 (DEFAULT_TABLE table=test-table [id=5b037aa46f0a40c788f416b907316e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:10.815212 22031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309776c3bc864cfe84bad4160ea69449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:10.818593 22259 tablet_service.cc:1467] Processing CreateTablet for tablet 309776c3bc864cfe84bad4160ea69449 (DEFAULT_TABLE table=test-table [id=5b037aa46f0a40c788f416b907316e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:10.819713 22259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309776c3bc864cfe84bad4160ea69449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:10.833925 22406 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Bootstrap starting.
I20260806 22:30:10.839675 22406 tablet_bootstrap.cc:654] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:10.841141 22406 log.cc:824] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:10.842494 22407 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Bootstrap starting.
I20260806 22:30:10.846349 22407 tablet_bootstrap.cc:654] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:10.846530 22406 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: No bootstrap required, opened a new log
I20260806 22:30:10.846868 22406 ts_tablet_manager.cc:1397] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260806 22:30:10.847781 22407 log.cc:824] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:10.857023 22408 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Bootstrap starting.
I20260806 22:30:10.858064 22407 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: No bootstrap required, opened a new log
I20260806 22:30:10.858404 22407 ts_tablet_manager.cc:1397] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260806 22:30:10.861085 22408 tablet_bootstrap.cc:654] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:10.862385 22408 log.cc:824] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:10.863376 22406 raft_consensus.cc:348] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.863996 22406 raft_consensus.cc:374] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:10.864159 22406 raft_consensus.cc:729] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8777ff4f350488da8362bcf425dbd28, State: Initialized, Role: FOLLOWER
I20260806 22:30:10.864981 22406 consensus_queue.cc:260] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.868508 22407 raft_consensus.cc:348] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.868969 22407 raft_consensus.cc:374] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:10.869099 22407 raft_consensus.cc:729] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13dc439193424ba1bc25da05d6a35683, State: Initialized, Role: FOLLOWER
I20260806 22:30:10.869763 22407 consensus_queue.cc:260] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.871268 21919 heartbeater.cc:503] Master 127.1.8.126:34063 was elected leader, sending a full tablet report...
I20260806 22:30:10.871958 22406 ts_tablet_manager.cc:1428] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Time spent starting tablet: real 0.025s user 0.010s sys 0.010s
I20260806 22:30:10.874217 22408 tablet_bootstrap.cc:492] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: No bootstrap required, opened a new log
I20260806 22:30:10.874579 22408 ts_tablet_manager.cc:1397] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.007s
I20260806 22:30:10.875363 22407 ts_tablet_manager.cc:1428] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260806 22:30:10.883785 22408 raft_consensus.cc:348] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.884140 22408 raft_consensus.cc:374] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:10.884235 22408 raft_consensus.cc:729] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be4d7598cec4c0fa22c2b7795600d47, State: Initialized, Role: FOLLOWER
I20260806 22:30:10.884799 22408 consensus_queue.cc:260] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.885895 22352 heartbeater.cc:503] Master 127.1.8.126:34063 was elected leader, sending a full tablet report...
I20260806 22:30:10.886323 22408 ts_tablet_manager.cc:1428] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
W20260806 22:30:10.932423 22353 tablet.cc:2369] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:10.940179 22414 raft_consensus.cc:482] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:30:10.940488 22414 raft_consensus.cc:504] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.942520 22414 leader_election.cc:288] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8777ff4f350488da8362bcf425dbd28 (127.1.8.65:33041), 13dc439193424ba1bc25da05d6a35683 (127.1.8.66:40755)
I20260806 22:30:10.949424 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.942686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:33041 (local address 127.1.8.67:59333)
0806 22:30:10.942843 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.942868 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.943000 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.943837 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.943842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.943867 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:10.944176 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.944195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.945653 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.945658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.947098 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.947111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.948702 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.948708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.948881 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.948919 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.949017 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.949144 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:30:10.949434 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.943247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:59333 (local address 127.1.8.65:33041)
0806 22:30:10.943477 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.943483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.943502 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:30:10.943581 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.943588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.943681 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.943776 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:10.944392 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.945512 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.947295 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.948536 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.948672 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.948766 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.949059 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.949193 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 22:30:10.950539 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.943555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:60629 (local address 127.1.8.66:40755)
0806 22:30:10.943758 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.943763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.944572 (+ 809us) server_negotiation.cc:407] Connection header received
0806 22:30:10.944716 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.944735 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.944808 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.944912 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:10.945465 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.946541 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.948374 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.949436 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.949574 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.949679 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:10.950186 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 22:30:10.950314 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:30:10.950598 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.943330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40755 (local address 127.1.8.67:60629)
0806 22:30:10.944449 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:10.944468 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:10.944588 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:10.944941 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:10.944945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:10.944970 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:10.945296 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.945307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.946695 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.946700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.948226 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:10.948238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:10.949812 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:10.949818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:10.949980 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:10.950033 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:30:10.950142 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:30:10.950260 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":761,"threads_started":1}
I20260806 22:30:10.950944 21841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309776c3bc864cfe84bad4160ea69449" candidate_uuid: "5be4d7598cec4c0fa22c2b7795600d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8777ff4f350488da8362bcf425dbd28" is_pre_election: true
I20260806 22:30:10.951568 21841 raft_consensus.cc:2454] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5be4d7598cec4c0fa22c2b7795600d47 in term 0.
I20260806 22:30:10.951974 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309776c3bc864cfe84bad4160ea69449" candidate_uuid: "5be4d7598cec4c0fa22c2b7795600d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13dc439193424ba1bc25da05d6a35683" is_pre_election: true
I20260806 22:30:10.952384 22184 leader_election.cc:302] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5be4d7598cec4c0fa22c2b7795600d47, b8777ff4f350488da8362bcf425dbd28; no voters:
I20260806 22:30:10.952576 22071 raft_consensus.cc:2454] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5be4d7598cec4c0fa22c2b7795600d47 in term 0.
I20260806 22:30:10.952809 22414 raft_consensus.cc:2781] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:30:10.952960 22414 raft_consensus.cc:482] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:30:10.953063 22414 raft_consensus.cc:3037] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:10.956171 22414 raft_consensus.cc:504] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.956830 22414 leader_election.cc:288] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [CANDIDATE]: Term 1 election: Requested vote from peers b8777ff4f350488da8362bcf425dbd28 (127.1.8.65:33041), 13dc439193424ba1bc25da05d6a35683 (127.1.8.66:40755)
I20260806 22:30:10.957214 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309776c3bc864cfe84bad4160ea69449" candidate_uuid: "5be4d7598cec4c0fa22c2b7795600d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13dc439193424ba1bc25da05d6a35683"
I20260806 22:30:10.957239 21841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309776c3bc864cfe84bad4160ea69449" candidate_uuid: "5be4d7598cec4c0fa22c2b7795600d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8777ff4f350488da8362bcf425dbd28"
I20260806 22:30:10.957398 22071 raft_consensus.cc:3037] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:10.957419 21841 raft_consensus.cc:3037] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:30:10.959548 22143 tablet.cc:2369] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:10.959827 22071 raft_consensus.cc:2454] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5be4d7598cec4c0fa22c2b7795600d47 in term 1.
I20260806 22:30:10.960408 22183 leader_election.cc:302] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13dc439193424ba1bc25da05d6a35683, 5be4d7598cec4c0fa22c2b7795600d47; no voters:
I20260806 22:30:10.960723 22414 raft_consensus.cc:2781] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:10.960832 21841 raft_consensus.cc:2454] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5be4d7598cec4c0fa22c2b7795600d47 in term 1.
I20260806 22:30:10.961452 22414 raft_consensus.cc:686] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 1 LEADER]: Becoming Leader. State: Replica: 5be4d7598cec4c0fa22c2b7795600d47, State: Running, Role: LEADER
I20260806 22:30:10.961956 22414 consensus_queue.cc:237] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } }
I20260806 22:30:10.967010 21601 catalog_manager.cc:5495] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5be4d7598cec4c0fa22c2b7795600d47 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "5be4d7598cec4c0fa22c2b7795600d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5be4d7598cec4c0fa22c2b7795600d47" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 } health_report { overall_health: UNKNOWN } } }
I20260806 22:30:11.005793 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:10.998798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.1.8.67:36451)
0806 22:30:10.999025 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:10.999032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:10.999051 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:30:10.999159 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:10.999164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:10.999297 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:10.999496 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:30:10.999955 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.001030 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.003097 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.003698 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.003823 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:11.004444 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:30:11.005155 (+ 711us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:30:11.005301 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:11.005440 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:30:11.005528 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:30:11.015774 21841 raft_consensus.cc:1261] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 1 FOLLOWER]: Refusing update from remote peer 5be4d7598cec4c0fa22c2b7795600d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:11.015807 22071 raft_consensus.cc:1261] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 1 FOLLOWER]: Refusing update from remote peer 5be4d7598cec4c0fa22c2b7795600d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:11.016948 22418 consensus_queue.cc:1035] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13dc439193424ba1bc25da05d6a35683" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:11.017362 22414 consensus_queue.cc:1035] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8777ff4f350488da8362bcf425dbd28" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 33041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:30:11.022055 21921 tablet.cc:2369] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:11.036713 22422 mvcc.cc:205] Tried to move back new op lower bound from 7315682963509379072 to 7315682963304050688. Current Snapshot: MvccSnapshot[applied={T|T < 7315682963509379072}]
I20260806 22:30:11.042865 22424 mvcc.cc:205] Tried to move back new op lower bound from 7315682963509379072 to 7315682963304050688. Current Snapshot: MvccSnapshot[applied={T|T < 7315682963509379072}]
I20260806 22:30:11.063756 21601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:11.064072 21601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:11.067695 21601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0948e9d16aba4f7393a1634ff87ee0f5: Sending DeleteTablet for 3 replicas of tablet 309776c3bc864cfe84bad4160ea69449
I20260806 22:30:11.069063 22259 tablet_service.cc:1514] Processing DeleteTablet for tablet 309776c3bc864cfe84bad4160ea69449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:11 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:60536
I20260806 22:30:11.069391 22031 tablet_service.cc:1514] Processing DeleteTablet for tablet 309776c3bc864cfe84bad4160ea69449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:11 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:54308
I20260806 22:30:11.070127 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet 309776c3bc864cfe84bad4160ea69449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:11 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:46482
I20260806 22:30:11.076754 22443 tablet_replica.cc:330] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: stopping tablet replica
I20260806 22:30:11.077306 22443 raft_consensus.cc:2229] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:11.077482 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 21658
I20260806 22:30:11.077953 22443 raft_consensus.cc:2258] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:11.087170 22443 ts_tablet_manager.cc:1905] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:11.093569 22441 tablet_replica.cc:330] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: stopping tablet replica
I20260806 22:30:11.093636 22440 tablet_replica.cc:330] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: stopping tablet replica
I20260806 22:30:11.094247 22441 raft_consensus.cc:2229] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:11.094241 22440 raft_consensus.cc:2229] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:30:11.095028 22440 raft_consensus.cc:2258] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:11.097429 22441 raft_consensus.cc:2258] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:11.097965 22440 ts_tablet_manager.cc:1905] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:11.100306 22441 ts_tablet_manager.cc:1905] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:11.100796 22443 ts_tablet_manager.cc:1918] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:11.101006 22443 log.cc:1196] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/309776c3bc864cfe84bad4160ea69449
I20260806 22:30:11.101506 22443 ts_tablet_manager.cc:1939] T 309776c3bc864cfe84bad4160ea69449 P b8777ff4f350488da8362bcf425dbd28: Deleting consensus metadata
I20260806 22:30:11.103698 21579 catalog_manager.cc:4841] TS b8777ff4f350488da8362bcf425dbd28 (127.1.8.65:33041): tablet 309776c3bc864cfe84bad4160ea69449 (table test-table [id=5b037aa46f0a40c788f416b907316e3c]) successfully deleted
I20260806 22:30:11.106745 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:11.086351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.1.8.65:33041)
0806 22:30:11.086542 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:11.086549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:11.086566 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:30:11.086629 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:11.086633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:11.086708 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:11.086830 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:11.088829 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.089890 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.091895 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.092565 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.092694 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:11.096617 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:11.096635 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:11.096658 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:11.096714 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:11.102010 (+ 5296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.103001 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.103011 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.103016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.103204 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.104635 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.104643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.104649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.105108 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:11.105450 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:11.105752 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 22:30:11.106013 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:30:11.107599 21910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:11.108183 22441 ts_tablet_manager.cc:1918] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:11.108417 22441 log.cc:1196] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/309776c3bc864cfe84bad4160ea69449
I20260806 22:30:11.109004 22441 ts_tablet_manager.cc:1939] T 309776c3bc864cfe84bad4160ea69449 P 13dc439193424ba1bc25da05d6a35683: Deleting consensus metadata
I20260806 22:30:11.110958 21578 catalog_manager.cc:4841] TS 13dc439193424ba1bc25da05d6a35683 (127.1.8.66:40755): tablet 309776c3bc864cfe84bad4160ea69449 (table test-table [id=5b037aa46f0a40c788f416b907316e3c]) successfully deleted
I20260806 22:30:11.114668 22440 ts_tablet_manager.cc:1918] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:11.114877 22440 log.cc:1196] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/309776c3bc864cfe84bad4160ea69449
I20260806 22:30:11.115358 22440 ts_tablet_manager.cc:1939] T 309776c3bc864cfe84bad4160ea69449 P 5be4d7598cec4c0fa22c2b7795600d47: Deleting consensus metadata
I20260806 22:30:11.117347 21579 catalog_manager.cc:4841] TS 5be4d7598cec4c0fa22c2b7795600d47 (127.1.8.67:36451): tablet 309776c3bc864cfe84bad4160ea69449 (table test-table [id=5b037aa46f0a40c788f416b907316e3c]) successfully deleted
I20260806 22:30:11.219660 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 21658
I20260806 22:30:11.246737 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 21925
I20260806 22:30:11.264904 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:11.247627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.1.8.66:40755)
0806 22:30:11.247812 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:11.247816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:11.247943 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:30:11.248174 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:11.248179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:11.248250 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:11.248338 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:11.250397 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.251224 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.253548 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.253975 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.254080 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:11.257994 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:11.258010 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:11.258027 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:11.258073 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:11.261533 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.262433 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.262440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.262442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.262616 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.263094 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.263098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.263100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.263476 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:11.263837 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:11.264206 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 22:30:11.264474 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:30:11.265870 22136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:11.367494 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 21925
I20260806 22:30:11.391491 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 22147
I20260806 22:30:11.406472 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:11.392655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.1.8.67:36451)
0806 22:30:11.392830 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:11.392834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:11.392850 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:30:11.392931 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:11.392935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:11.393005 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:11.393124 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:11.394407 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.395192 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.396585 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:11.397092 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:11.397229 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:11.399905 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:11.399917 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:11.399933 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:11.399970 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:11.403316 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.404087 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.404143 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.404147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.404329 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:11.404749 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:11.404754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:11.404759 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:11.405149 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:11.405546 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:11.405825 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:30:11.406092 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:30:11.407327 22346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:11.529238 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 22147
I20260806 22:30:11.559209 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 21549
I20260806 22:30:11.560437 21648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:11.674922 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 21549
I20260806 22:30:11.704003 1057 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:30:12.050818 1057 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:30:12.051023 1057 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:30:12.407799 1057 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:30:12.408 UTC [20136] LOG: received smart shutdown request
2026-08-06 22:30:12.411 UTC [20136] LOG: background worker "logical replication launcher" (PID 20150) exited with exit code 1
2026-08-06 22:30:12.411 UTC [20141] LOG: shutting down
2026-08-06 22:30:12.575 UTC [20136] LOG: database system is shut down
2026-08-06T22:30:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54525 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:30:12 dist-test-slave-7vjv krb5kdc[22465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:30:12 dist-test-slave-7vjv krb5kdc[22465](info): set up 2 sockets
Aug 06 22:30:12 dist-test-slave-7vjv krb5kdc[22465](info): commencing operation
krb5kdc: starting...
W20260806 22:30:14.021399 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:30:14 dist-test-slave-7vjv krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:30:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:30:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:30:14.307264 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42823
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:14.671525 22484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:14.671800 22484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:14.671926 22484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:14.679167 22484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:14.679266 22484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:14.679368 22484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:14.679472 22484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:14.679555 22484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:14.689229 22484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:14.690570 22484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:14.691852 22484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:14.703752 22495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:14.703910 22494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:14.703891 22497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:14.704720 22484 server_base.cc:1034] running on GCE node
I20260806 22:30:14.705509 22484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:14.706856 22484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:14.708128 22484 hybrid_clock.cc:648] HybridClock initialized: now 1786055414708051 us; error 53 us; skew 500 ppm
I20260806 22:30:14.708477 22484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:14 dist-test-slave-7vjv krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:14.716557 22484 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:14.718070 22484 webserver.cc:460] Webserver started at http://127.1.8.126:33051/ using document root <none> and password file <none>
I20260806 22:30:14.718736 22484 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:14.718839 22484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:14.719108 22484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:14.722127 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "9cca1232b7294120b53d2332fadcefc7"
format_stamp: "Formatted at 2026-08-06 22:30:14 on dist-test-slave-7vjv"
server_key: "eed66940293aedb8750331478e1f1037"
server_key_iv: "bcf74c861aa8c57acf71f5d77c4f8760"
server_key_version: "encryptionkey@0"
I20260806 22:30:14.722884 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "9cca1232b7294120b53d2332fadcefc7"
format_stamp: "Formatted at 2026-08-06 22:30:14 on dist-test-slave-7vjv"
server_key: "eed66940293aedb8750331478e1f1037"
server_key_iv: "bcf74c861aa8c57acf71f5d77c4f8760"
server_key_version: "encryptionkey@0"
I20260806 22:30:14.727882 22484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:30:14.731299 22508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:14.732606 22484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:30:14.732745 22484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "9cca1232b7294120b53d2332fadcefc7"
format_stamp: "Formatted at 2026-08-06 22:30:14 on dist-test-slave-7vjv"
server_key: "eed66940293aedb8750331478e1f1037"
server_key_iv: "bcf74c861aa8c57acf71f5d77c4f8760"
server_key_version: "encryptionkey@0"
I20260806 22:30:14.732916 22484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:14.747288 22484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:14.751631 22484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:14.751897 22484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:14.772192 22484 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42823
I20260806 22:30:14.772212 22587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42823 every 8 connection(s)
I20260806 22:30:14.773707 22484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:14.777374 22588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:14.777595 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 22484
I20260806 22:30:14.777825 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:30:14.778548 1057 external_mini_cluster.cc:1428] Setting key c4fc436a0310c7925f291b6da4353a1d
I20260806 22:30:14.783949 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:30:14 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.8.125@KRBTEST.COM; defaulting to no policy
I20260806 22:30:14.808216 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.787094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42823 (local address 127.0.0.1:44362)
0806 22:30:14.788295 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.788331 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:14.790035 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:14.790920 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:14.790930 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:14.791797 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:14.792312 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:14.792344 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:14.793904 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:14.793912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:14.795841 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:14.795858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:14.796914 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:14.796927 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:14.797157 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:14.797925 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:14.797958 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:14.800710 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:14.803909 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:14.803921 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:14.803933 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:14.804452 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:14.804873 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:14.804878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:14.804882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:14.805036 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:14.805797 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:14.805809 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:14.806037 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 22:30:14.806264 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 22:30:14.806854 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":143,"thread_start_us":739,"threads_started":1}
I20260806 22:30:14.808235 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.787093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.1.8.126:42823)
0806 22:30:14.787956 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:14.787975 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:14.788477 (+ 502us) server_negotiation.cc:407] Connection header received
0806 22:30:14.790448 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:14.790454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:14.790675 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:14.790828 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:14.792536 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:14.793731 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:14.795994 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:14.796666 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:14.796795 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:14.800932 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:14.800939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:14.800950 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:14.800978 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:14.803771 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:14.804592 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:14.804599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:14.804601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:14.804742 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:14.805153 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:14.805161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:14.805164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:14.805649 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:14.805878 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:14.806410 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 22:30:14.806710 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":408,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.125@KRBTEST.COM" created.
I20260806 22:30:14.811316 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:30:14.812405 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.811968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37744)
0806 22:30:14.812185 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.812291 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 22:30:14.812633 22515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
I20260806 22:30:14.814204 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.813851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37746)
0806 22:30:14.814013 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.814075 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:30:14.814397 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:46089: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
I20260806 22:30:14.843025 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:46089
--webserver_interface=127.1.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:14.863726 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } attempt: 1
I20260806 22:30:14.864946 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.864505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37758)
0806 22:30:14.864704 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.864810 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:30:14.865934 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.865581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37768)
0806 22:30:14.865730 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.865805 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:30:14.866137 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:46089: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
I20260806 22:30:14.934468 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } attempt: 2
I20260806 22:30:14.935690 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.935204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37778)
0806 22:30:14.935454 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.935552 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 22:30:14.936846 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:14.936488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37786)
0806 22:30:14.936652 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:14.936718 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 22:30:14.937132 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:46089: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
I20260806 22:30:15.028472 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } attempt: 3
I20260806 22:30:15.029708 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.029266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37798)
0806 22:30:15.029465 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.029565 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:30:15.030684 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.030330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37804)
0806 22:30:15.030484 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.030567 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:30:15.030869 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:46089: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
I20260806 22:30:15.174095 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } attempt: 4
I20260806 22:30:15.175258 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.174741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37820)
0806 22:30:15.175040 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.175128 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 22:30:15.176301 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.175882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37836)
0806 22:30:15.176030 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.176182 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 22:30:15.176474 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:46089: Network error: Client connection negotiation failed: client connection to 127.1.8.125:46089: connect: Connection refused (error 111)
W20260806 22:30:15.178010 22602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:15.178325 22602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:15.178490 22602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:15.185750 22602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:15.185894 22602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:15.185985 22602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:15.186091 22602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:15.186174 22602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:15.195858 22602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:15.197196 22602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:15.198499 22602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:15.210263 22608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:15.210295 22609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:15.210224 22611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:15.211402 22602 server_base.cc:1034] running on GCE node
I20260806 22:30:15.212293 22602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:15.213603 22602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:15.214803 22602 hybrid_clock.cc:648] HybridClock initialized: now 1786055415214762 us; error 29 us; skew 500 ppm
I20260806 22:30:15.215159 22602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:15.218437 22602 init.cc:377] Logged in from keytab as kudu/127.1.8.125@KRBTEST.COM (short username kudu)
I20260806 22:30:15.219947 22602 webserver.cc:460] Webserver started at http://127.1.8.125:33411/ using document root <none> and password file <none>
I20260806 22:30:15.220664 22602 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:15.220795 22602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:15.221145 22602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:15.224143 22602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data/instance:
uuid: "711644524d69476e84ffe8131d419a4f"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "665ad87abdbe2c21fe76a7eaf635b9b2"
server_key_iv: "18498f2c46ab460f00ffa602a5d29568"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.224997 22602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal/instance:
uuid: "711644524d69476e84ffe8131d419a4f"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "665ad87abdbe2c21fe76a7eaf635b9b2"
server_key_iv: "18498f2c46ab460f00ffa602a5d29568"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.230049 22602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:30:15.233479 22625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:15.234788 22602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:30:15.234922 22602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "711644524d69476e84ffe8131d419a4f"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "665ad87abdbe2c21fe76a7eaf635b9b2"
server_key_iv: "18498f2c46ab460f00ffa602a5d29568"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.235076 22602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:15.267818 22602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:15.271883 22602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:15.272197 22602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:15.290530 22602 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.125:46089
I20260806 22:30:15.290573 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.125:46089 every 8 connection(s)
I20260806 22:30:15.291962 22602 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
I20260806 22:30:15.295655 22714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:15.299726 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 22602
I20260806 22:30:15.299932 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-1/wal/instance
I20260806 22:30:15.300531 1057 external_mini_cluster.cc:1428] Setting key 4c70f2509794060bd45c8dc0dc1f9398
I20260806 22:30:15.303535 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:15.327266 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.306984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.1.8.126:42823)
0806 22:30:15.307217 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.307225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.308400 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 22:30:15.310192 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.310201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.310294 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.310438 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.311829 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.312900 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.315420 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.315841 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.315939 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.320014 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.320023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.320030 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.320079 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.323917 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.324701 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.324709 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.324715 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.324909 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.325323 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.325331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.325337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.325727 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.325910 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.326539 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.326816 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:30:15.327812 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.307371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42823 (local address 127.0.0.1:44376)
0806 22:30:15.308227 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.308281 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.309939 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.310618 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.310626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.311140 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.311664 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.311679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.313067 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.313074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.315197 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.315215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.316067 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.316075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.316424 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.317125 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.317157 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.319785 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.324068 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.324076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.324087 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.324579 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.325041 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.325044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.325046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.325204 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.325847 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.325853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.326154 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.326379 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.326747 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":378,"threads_started":1}
I20260806 22:30:15.329595 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.124@KRBTEST.COM" created.
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:30:15.345415 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.330497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37846)
0806 22:30:15.330659 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.330677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.330796 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.331610 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.331614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.332012 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.332193 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.332201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.333705 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.333710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.334699 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.334711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.335623 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.335629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.335851 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.336469 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.336486 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.338544 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.343048 (+ 4504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.343052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.343055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.343411 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.343829 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.343833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.343835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.343958 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.344770 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.344773 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.344889 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.344995 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.345183 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:30:15.345867 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.330577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.1.8.125:46089)
0806 22:30:15.331238 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.331244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.331267 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:30:15.331308 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.331312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.331460 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.331611 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.332340 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.333552 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.334827 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.335459 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.335607 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.338819 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.338835 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.338853 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.338897 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.342873 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.343537 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.343548 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.343554 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.343699 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.344064 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.344071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.344077 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.344618 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.344849 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.345167 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.345446 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":326,"threads_started":1}
I20260806 22:30:15.347971 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:30:15.349007 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.348610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60846)
0806 22:30:15.348762 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.348889 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:30:15.349236 22632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.350781 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.350470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60848)
0806 22:30:15.350620 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.350693 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:30:15.351040 22714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
I20260806 22:30:15.368571 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:35987
--webserver_interface=127.1.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:15.400352 22714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 1
I20260806 22:30:15.401599 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.401143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60862)
0806 22:30:15.401340 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.401447 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 22:30:15.402611 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.402265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60866)
0806 22:30:15.402417 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.402506 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:30:15.402817 22714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.471060 22714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 2
I20260806 22:30:15.472393 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.471831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60874)
0806 22:30:15.472058 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.472215 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 22:30:15.473452 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.473108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60876)
0806 22:30:15.473264 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.473336 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:30:15.473661 22714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.475744 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } attempt: 5
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:30:15.489742 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.476605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.1.8.125:46089)
0806 22:30:15.476768 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.476775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.476793 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:30:15.476905 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.476910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.476994 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.477101 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.477917 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.478667 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.479881 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.480436 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.480570 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.484390 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.484395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.484398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.484429 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.486936 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.487840 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.487843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.487846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.487958 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.488428 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.488432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.488434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.488731 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.488879 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.489362 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.489528 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:30:15.490123 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.476426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37852)
0806 22:30:15.476599 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.476624 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.476764 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.477141 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.477147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.477587 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.477762 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.477772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.478839 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.478846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.479731 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.479742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.480595 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.480601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.480812 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.481557 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.481576 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.484178 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.487102 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.487111 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.487117 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.487722 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.488073 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.488078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.488082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.488300 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.488875 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.488881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.489084 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.489288 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.489599 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:30:15.491189 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:30:15.492275 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.491830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60890)
0806 22:30:15.491987 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.492066 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 22:30:15.493269 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.492974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60896)
0806 22:30:15.493111 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.493187 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:30:15.493505 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.544745 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 1
I20260806 22:30:15.546070 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.545558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60904)
0806 22:30:15.545778 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.545881 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:30:15.547079 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.546762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60914)
0806 22:30:15.546918 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.546982 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:30:15.547317 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.564881 22714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 3
I20260806 22:30:15.566118 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.565637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60920)
0806 22:30:15.565856 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.565958 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:30:15.567361 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.566896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60926)
0806 22:30:15.567163 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.567241 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260806 22:30:15.567592 22714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.615578 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 2
I20260806 22:30:15.617017 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.616438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60932)
0806 22:30:15.616682 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.616806 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:30:15.618047 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.617701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60944)
0806 22:30:15.617869 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.617950 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:30:15.618290 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.710857 22714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 4
I20260806 22:30:15.712044 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.711548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60960)
0806 22:30:15.711789 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.711882 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:30:15.713114 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.712776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60972)
0806 22:30:15.712935 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.713016 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:30:15.713333 22714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
W20260806 22:30:15.715966 22721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:15.716316 22721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:15.716487 22721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:15.723793 22721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:15.723910 22721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:15.723995 22721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:15.724124 22721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:15.724203 22721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:15.724506 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 3
I20260806 22:30:15.725476 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.725034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60986)
0806 22:30:15.725244 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.725330 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:30:15.726490 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.726141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:60998)
0806 22:30:15.726318 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.726394 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:30:15.726703 22588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35987: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35987: connect: Connection refused (error 111)
I20260806 22:30:15.734328 22721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42823,127.1.8.125:46089,127.1.8.124:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:15.735623 22721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:15.737123 22721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:15.750295 22730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:15.750247 22731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:15.750321 22734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:15.751396 22721 server_base.cc:1034] running on GCE node
I20260806 22:30:15.752344 22721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:15.753821 22721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:15.755054 22721 hybrid_clock.cc:648] HybridClock initialized: now 1786055415755011 us; error 56 us; skew 500 ppm
I20260806 22:30:15.755398 22721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:15.758606 22721 init.cc:377] Logged in from keytab as kudu/127.1.8.124@KRBTEST.COM (short username kudu)
I20260806 22:30:15.760039 22721 webserver.cc:460] Webserver started at http://127.1.8.124:46297/ using document root <none> and password file <none>
I20260806 22:30:15.760761 22721 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:15.760926 22721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:15.761234 22721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:15.764262 22721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data/instance:
uuid: "5f9193794c6a46fa965f984769afca3c"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "4d7b682318d240e7fb3ab1989711ca3e"
server_key_iv: "660a5935b2b2b5b52b1de2a582476397"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.765103 22721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal/instance:
uuid: "5f9193794c6a46fa965f984769afca3c"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "4d7b682318d240e7fb3ab1989711ca3e"
server_key_iv: "660a5935b2b2b5b52b1de2a582476397"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.770294 22721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:30:15.773973 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:15.775478 22721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:30:15.775664 22721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "5f9193794c6a46fa965f984769afca3c"
format_stamp: "Formatted at 2026-08-06 22:30:15 on dist-test-slave-7vjv"
server_key: "4d7b682318d240e7fb3ab1989711ca3e"
server_key_iv: "660a5935b2b2b5b52b1de2a582476397"
server_key_version: "encryptionkey@0"
I20260806 22:30:15.775846 22721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:15.795612 22721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:15.800148 22721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:15.800464 22721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:15.820117 22721 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.124:35987
I20260806 22:30:15.820205 22820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.124:35987 every 8 connection(s)
I20260806 22:30:15.821522 22721 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
I20260806 22:30:15.825201 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 22721
I20260806 22:30:15.825394 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055191034056-1057-0/minicluster-data/master-2/wal/instance
I20260806 22:30:15.825433 22821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:15.826058 1057 external_mini_cluster.cc:1428] Setting key 6751420932f86acdd1109bb2bd3be014
I20260806 22:30:15.835209 22821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:15.850786 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.828077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.1.8.124:35987)
0806 22:30:15.829019 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.829030 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.829071 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:30:15.830550 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.830571 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.830926 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.831250 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.833088 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.834339 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.836353 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.837097 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.837240 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.842862 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.842869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.842878 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.842915 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.846032 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.846844 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.846850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.846856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.847061 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.847498 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.847501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.847503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.848210 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.848516 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.848961 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.849356 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":425,"threads_started":1}
I20260806 22:30:15.854115 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.838145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42823 (local address 127.0.0.1:44382)
0806 22:30:15.838932 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.838983 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.839135 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.839574 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.839578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.840281 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.840541 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.840551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.841852 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.841858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.843245 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.843264 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.844279 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.844284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.844402 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.845036 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.845064 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.847494 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.851354 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.851366 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.851372 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.851790 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.852295 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.852299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.852303 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.852453 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.853082 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.853089 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.853279 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.853435 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.853733 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":415,"threads_started":1}
I20260806 22:30:15.854277 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.838131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.1.8.126:42823)
0806 22:30:15.838728 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.838734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.839078 (+ 344us) server_negotiation.cc:407] Connection header received
0806 22:30:15.839271 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.839277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.839375 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.839502 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.840693 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.841685 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.843402 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.844050 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.844222 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.847701 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.847706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.847710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.847748 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.851219 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.851968 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.851975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.851979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.852177 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.852567 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.852574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.852577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.852879 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.853059 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.853589 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.853810 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":323,"threads_started":1}
I20260806 22:30:15.855789 22821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:30:15.869731 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.856668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:46089 (local address 127.0.0.1:37868)
0806 22:30:15.856861 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.856880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.856977 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.857330 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.857334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.857811 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.857981 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.857995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.858962 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.858965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.860171 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.860186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.860933 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.860941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.861156 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.861666 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.861692 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.864318 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.867263 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.867266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.867269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.867626 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.868051 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.868054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.868056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.868297 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.868823 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.868827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.868976 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.869130 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.869404 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 22:30:15.869751 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.853901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.1.8.126:42823)
0806 22:30:15.854561 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.854568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.854589 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:30:15.854654 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.854661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.854776 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.854893 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.856710 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.857429 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.858868 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.859329 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.859420 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.864235 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.864242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.864247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.864284 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.867073 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.867747 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.867753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.867757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.867874 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.868318 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.868324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.868330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.868645 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.868908 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.869191 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.869392 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":970,"threads_started":1}
I20260806 22:30:15.870036 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.856711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.1.8.125:46089)
0806 22:30:15.856914 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.856920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.856938 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:30:15.857135 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.857141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.857211 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.857310 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.858136 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.858764 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.860325 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.860757 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.860863 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.864510 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.864515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.864518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.864550 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.867094 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.867747 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.867753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.867758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.867907 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.868410 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.868414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.868416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.868671 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.868835 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.869260 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.869481 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:30:15.870991 22821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:30:15.885497 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.871578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:32782)
0806 22:30:15.871736 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.871753 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.871849 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.872319 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.872322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.872647 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.872763 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.872770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.874127 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.874132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.875438 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.875450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.876444 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.876448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.876572 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.877118 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.877141 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.879762 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.882818 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.882825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.882829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.883204 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.883574 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.883579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.883582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.883710 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.884487 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.884493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.884658 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.884828 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.885108 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:30:15.886436 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.871812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.1.8.124:35987)
0806 22:30:15.871974 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.871978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.871992 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:30:15.872028 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.872031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.872078 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.872200 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.872908 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.873996 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.875578 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.876288 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.876410 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.879934 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.879940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.879945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.879980 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.882693 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.883317 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.883324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.883327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.883460 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.883864 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.883870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.883874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.884269 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.884405 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.885857 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.886097 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:30:15.889106 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.871647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.1.8.125:46089)
0806 22:30:15.871815 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.871820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.872160 (+ 340us) server_negotiation.cc:407] Connection header received
0806 22:30:15.872335 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.872340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.872406 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.872500 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.873979 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.875096 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.876723 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.877358 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.877496 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.882413 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.882425 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.882429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.882457 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.885777 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.886896 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.886921 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.886924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.887103 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.887550 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.887557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.887563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.887937 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.888253 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.888537 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.888770 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:30:15.892650 22821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: Bootstrap starting.
I20260806 22:30:15.894244 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:30:15.896723 22821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:15.898134 22821 log.cc:824] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:15.901366 22821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: No bootstrap required, opened a new log
I20260806 22:30:15.903918 22588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 4
I20260806 22:30:15.919592 22821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.920130 22821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:15.920267 22821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9193794c6a46fa965f984769afca3c, State: Initialized, Role: FOLLOWER
I20260806 22:30:15.921059 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.900232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.1.8.126:42823)
0806 22:30:15.900561 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.900570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.901780 (+ 1210us) server_negotiation.cc:407] Connection header received
0806 22:30:15.902053 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.902058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.902161 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.902241 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.905224 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.906298 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.908369 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.909027 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.909124 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.913010 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.913017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.913021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.913062 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.916871 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.917902 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.917908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.917911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.918142 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.918648 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.918654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.918657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.919339 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.919631 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.920340 (+ 709us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.920603 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
Aug 06 22:30:15 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:30:15.922648 22821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.927613 22821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:15.931432 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:15.932049 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:15.933380 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.900466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37888 (local address 127.1.8.125:46089)
0806 22:30:15.900659 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.900664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.912671 (+ 12007us) server_negotiation.cc:407] Connection header received
0806 22:30:15.912941 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.912947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.913027 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.913118 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.915154 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.916315 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.920742 (+ 4427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.921387 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.921482 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.925672 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.925678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.925682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.925723 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.929245 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.930284 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.930292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.930298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.930511 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.931038 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.931044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.931047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.931471 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.931797 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.932531 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.932857 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:30:15.933466 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.904530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:32796)
0806 22:30:15.904931 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:15.904953 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:15.905072 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:15.912618 (+ 7546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:15.912626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:15.913314 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:15.913601 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.913618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.915799 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.915805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.917399 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:15.917417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.918451 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.918458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:15.918635 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.919192 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:15.919212 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:15.922443 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:15.929481 (+ 7038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.929488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.929492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.930007 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.930470 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:15.930478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:15.930484 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:15.930667 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:15.932444 (+ 1777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:15.932450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:15.932611 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 22:30:15.932742 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:30:15.933060 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260806 22:30:15.934821 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.901938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.1.8.124:35987)
0806 22:30:15.902209 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.902218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.915683 (+ 13465us) server_negotiation.cc:407] Connection header received
0806 22:30:15.916070 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.916075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.916156 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.916217 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.918311 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.919302 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.921406 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.922090 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.922182 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.926273 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.926542 (+ 269us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.926550 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.926817 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.931108 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.932065 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.932073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.932076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.932244 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.932712 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.932720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.932725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.933083 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.933318 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.934051 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.934267 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:30:15.935596 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:15.905306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.1.8.124:35987)
0806 22:30:15.911079 (+ 5773us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:15.911088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:15.911118 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:30:15.911186 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:15.911194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:15.911285 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:15.913790 (+ 2505us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:15.914145 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.915643 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.917605 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:15.918272 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:15.918932 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:15.924662 (+ 5730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:15.924669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:15.924674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:15.924713 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:15.929345 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.930162 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.930169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.930173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.930333 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:15.930825 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:15.930832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:15.930838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:15.932277 (+ 1439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:15.934492 (+ 2215us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:15.934613 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:30:15.934828 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5596,"thread_start_us":5356,"threads_started":1}
I20260806 22:30:15.937065 22841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:30:15.937376 22841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.938853 22841 leader_election.cc:288] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cca1232b7294120b53d2332fadcefc7 (127.1.8.126:42823), 711644524d69476e84ffe8131d419a4f (127.1.8.125:46089)
I20260806 22:30:15.942692 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: Bootstrap starting.
I20260806 22:30:15.947201 22588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:15.948573 22588 log.cc:824] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: Log is configured to *not* fsync() on all Append() calls
W20260806 22:30:15.962059 22862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:30:15.962180 22862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:30:15.947496 22721 master_runner.cc:186] Error getting master registration for 127.1.8.126:42823: OK, instance_id { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" instance_seqno: 1786055414766388 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:30:15.965157 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: No bootstrap required, opened a new log
I20260806 22:30:15.965337 22721 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:30:15.976564 22588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.976948 22588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:15.977092 22588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cca1232b7294120b53d2332fadcefc7, State: Initialized, Role: FOLLOWER
I20260806 22:30:15.977777 22588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.979426 22866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:15.979905 22866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:15.980926 22588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:15.980988 22551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f9193794c6a46fa965f984769afca3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cca1232b7294120b53d2332fadcefc7" is_pre_election: true
I20260806 22:30:15.981580 22551 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9193794c6a46fa965f984769afca3c in term 0.
I20260806 22:30:15.982250 22753 leader_election.cc:302] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9193794c6a46fa965f984769afca3c, 9cca1232b7294120b53d2332fadcefc7; no voters:
I20260806 22:30:15.982780 22841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:30:15.982959 22841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:30:15.983088 22841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:15.987393 22841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:15.988668 22551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f9193794c6a46fa965f984769afca3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cca1232b7294120b53d2332fadcefc7"
I20260806 22:30:15.988854 22551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:15.992264 22551 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9193794c6a46fa965f984769afca3c in term 1.
I20260806 22:30:15.992697 22753 leader_election.cc:302] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9193794c6a46fa965f984769afca3c, 9cca1232b7294120b53d2332fadcefc7; no voters:
I20260806 22:30:15.993374 22841 leader_election.cc:288] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [CANDIDATE]: Term 1 election: Requested vote from peers 9cca1232b7294120b53d2332fadcefc7 (127.1.8.126:42823), 711644524d69476e84ffe8131d419a4f (127.1.8.125:46089)
I20260806 22:30:15.993666 22841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:15.993847 22841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [term 1 LEADER]: Becoming Leader. State: Replica: 5f9193794c6a46fa965f984769afca3c, State: Running, Role: LEADER
I20260806 22:30:15.994498 22841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:16.001709 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9193794c6a46fa965f984769afca3c. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.002113 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: This master's current role is: LEADER
W20260806 22:30:16.004745 22517 tablet.cc:2369] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:16.008773 22887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:16.011518 22887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:16.012490 22714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } attempt: 5
Aug 06 22:30:16 dist-test-slave-7vjv krb5kdc[22465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:30:16.032423 22551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [term 1 FOLLOWER]: Refusing update from remote peer 5f9193794c6a46fa965f984769afca3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:16.033589 22885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:16.038401 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:16.013051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35987 (local address 127.0.0.1:32808)
0806 22:30:16.013397 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:16.013420 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:16.013542 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:16.014492 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:16.014499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:16.015017 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:16.015237 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:16.015252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:16.016914 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:16.016921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:16.018393 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:16.018409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:16.019889 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:16.019896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:16.020165 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:16.020900 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:16.020925 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:16.024431 (+ 3506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:16.031314 (+ 6883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:16.031319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:16.031352 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:16.033706 (+ 2354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:16.034565 (+ 859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:16.034570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:16.034574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:16.034886 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:16.036521 (+ 1635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:16.036555 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:16.036835 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 22:30:16.037131 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 22:30:16.037630 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260806 22:30:16.042279 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:16.013752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.1.8.124:35987)
0806 22:30:16.013929 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:16.013934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:16.013953 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:30:16.014008 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:16.014013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:16.014115 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:16.014215 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:16.015441 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:16.016709 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:16.018614 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:16.019476 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:16.019632 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:16.024704 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:16.024711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:16.024715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:16.024750 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:16.031087 (+ 6337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:16.034029 (+ 2942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:16.034037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:16.034040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:16.034331 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:16.035180 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:16.035187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:16.035192 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:16.036044 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:16.039115 (+ 3071us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:16.039837 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 22:30:16.040675 (+ 838us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:30:16.048236 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: Bootstrap starting.
W20260806 22:30:16.052606 22757 tablet.cc:2369] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:16.056727 22714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:16.058918 22714 log.cc:824] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:16.067541 22866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9193794c6a46fa965f984769afca3c. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.067903 22866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:16.068676 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: No bootstrap required, opened a new log
I20260806 22:30:16.078274 22888 mvcc.cc:205] Tried to move back new op lower bound from 7315682984034742272 to 7315682983919394816. Current Snapshot: MvccSnapshot[applied={T|T < 7315682984034742272}]
I20260806 22:30:16.071864 22484 master_runner.cc:186] Error getting master registration for 127.1.8.125:46089: OK, instance_id { permanent_uuid: "711644524d69476e84ffe8131d419a4f" instance_seqno: 1786055415285406 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:30:16.089922 22484 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:30:16.091653 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.092000 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:16.094581 22887 catalog_manager.cc:1347] Generated new cluster ID: a1007fabc29f4c6f903e55b79d0b0648
I20260806 22:30:16.094676 22887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:16.095148 22895 mvcc.cc:205] Tried to move back new op lower bound from 7315682984034742272 to 7315682983919394816. Current Snapshot: MvccSnapshot[applied={T|T < 7315682984034742272}]
I20260806 22:30:16.095605 22714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:16.095988 22714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:16.097918 22866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.098189 22866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:30:16.099457 22891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:30:16.099579 22891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:30:16.104157 22714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711644524d69476e84ffe8131d419a4f, State: Initialized, Role: FOLLOWER
I20260806 22:30:16.104954 22714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } }
I20260806 22:30:16.119038 22714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:16.119052 22669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f9193794c6a46fa965f984769afca3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711644524d69476e84ffe8131d419a4f"
I20260806 22:30:16.119427 22667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:16.120229 22670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f9193794c6a46fa965f984769afca3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711644524d69476e84ffe8131d419a4f" is_pre_election: true
I20260806 22:30:16.118336 22906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.120703 22906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:16.123793 22667 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [term 1 FOLLOWER]: Refusing update from remote peer 5f9193794c6a46fa965f984769afca3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:16.124642 22885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:16.149303 22887 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:30:16.151273 22887 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:16.161062 22885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.161396 22885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:16.171275 22887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f9193794c6a46fa965f984769afca3c: Generated new TSK 0
I20260806 22:30:16.172318 22887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:30:16.173664 22906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9193794c6a46fa965f984769afca3c. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.173977 22906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:16.174256 22913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f9193794c6a46fa965f984769afca3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cca1232b7294120b53d2332fadcefc7" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42823 } } peers { permanent_uuid: "711644524d69476e84ffe8131d419a4f" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 46089 } } peers { permanent_uuid: "5f9193794c6a46fa965f984769afca3c" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35987 } } }
I20260806 22:30:16.179034 22913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:30:16.199136 22928 catalog_manager.cc:1259] Loaded cluster ID: a1007fabc29f4c6f903e55b79d0b0648
I20260806 22:30:16.199267 22928 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: loading cluster ID for follower catalog manager: success
I20260806 22:30:16.202942 22928 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: acquiring CA information for follower catalog manager: success
I20260806 22:30:16.214329 22928 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 711644524d69476e84ffe8131d419a4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:30:17.101627 22891 catalog_manager.cc:1259] Loaded cluster ID: a1007fabc29f4c6f903e55b79d0b0648
I20260806 22:30:17.101737 22891 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: loading cluster ID for follower catalog manager: success
I20260806 22:30:17.104261 22891 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: acquiring CA information for follower catalog manager: success
I20260806 22:30:17.105998 22891 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9cca1232b7294120b53d2332fadcefc7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:30:17.340803 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 22484
I20260806 22:30:17.341944 22582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:17.452879 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 22484
I20260806 22:30:17.484568 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 22602
I20260806 22:30:17.485790 22705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:17.595129 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 22602
I20260806 22:30:17.625687 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 22721
I20260806 22:30:17.626909 22815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:17.735913 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 22721
2026-08-06T22:30:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5030 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:30:17 dist-test-slave-7vjv krb5kdc[22943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:30:17 dist-test-slave-7vjv krb5kdc[22943](info): set up 2 sockets
Aug 06 22:30:17 dist-test-slave-7vjv krb5kdc[22943](info): commencing operation
krb5kdc: starting...
W20260806 22:30:19.087600 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:30:19 dist-test-slave-7vjv krb5kdc[22943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:30:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:30:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:30:19.394199 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42029
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:19.753124 22961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:19.753417 22961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:19.753510 22961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:19.761296 22961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:19.761395 22961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:19.761448 22961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:19.761505 22961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:19.761551 22961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:19.771189 22961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:19.772647 22961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:19.773864 22961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:19.785363 22970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:19.785370 22969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:19.785939 22972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:19.786409 22961 server_base.cc:1034] running on GCE node
I20260806 22:30:19.787166 22961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:19.788563 22961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:19.789774 22961 hybrid_clock.cc:648] HybridClock initialized: now 1786055419789733 us; error 25 us; skew 500 ppm
I20260806 22:30:19.790127 22961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:19 dist-test-slave-7vjv krb5kdc[22943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055419, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:19.795797 22961 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:19.797389 22961 webserver.cc:460] Webserver started at http://127.1.8.126:34661/ using document root <none> and password file <none>
I20260806 22:30:19.798071 22961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:19.798218 22961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:19.798527 22961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:19.801489 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "8833652d1d354a62a673df60c4d564e5"
format_stamp: "Formatted at 2026-08-06 22:30:19 on dist-test-slave-7vjv"
server_key: "4c6b3e0e1885f1eb63a63e67dd7e3bc5"
server_key_iv: "31dd28096cc619271f8f8c6a33ae062a"
server_key_version: "encryptionkey@0"
I20260806 22:30:19.802256 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "8833652d1d354a62a673df60c4d564e5"
format_stamp: "Formatted at 2026-08-06 22:30:19 on dist-test-slave-7vjv"
server_key: "4c6b3e0e1885f1eb63a63e67dd7e3bc5"
server_key_iv: "31dd28096cc619271f8f8c6a33ae062a"
server_key_version: "encryptionkey@0"
I20260806 22:30:19.807252 22961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 22:30:19.810497 22983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:19.811702 22961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:30:19.811888 22961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "8833652d1d354a62a673df60c4d564e5"
format_stamp: "Formatted at 2026-08-06 22:30:19 on dist-test-slave-7vjv"
server_key: "4c6b3e0e1885f1eb63a63e67dd7e3bc5"
server_key_iv: "31dd28096cc619271f8f8c6a33ae062a"
server_key_version: "encryptionkey@0"
I20260806 22:30:19.812104 22961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:19.830899 22961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:19.835489 22961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:19.835811 22961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:19.854720 22961 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42029
I20260806 22:30:19.854763 23059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42029 every 8 connection(s)
I20260806 22:30:19.856048 22961 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:19.859676 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:19.863854 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 22961
I20260806 22:30:19.864173 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:30:19.865010 1057 external_mini_cluster.cc:1428] Setting key 6641142432afdbc1498c144df75411ef
I20260806 22:30:19.870965 23064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5: Bootstrap starting.
I20260806 22:30:19.874032 23064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:19.875013 23064 log.cc:824] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:19.878372 23064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5: No bootstrap required, opened a new log
Aug 06 22:30:19 dist-test-slave-7vjv krb5kdc[22943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:19.889158 23064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } }
I20260806 22:30:19.889497 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:19.866857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36228 (local address 127.1.8.126:42029)
0806 22:30:19.867637 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:19.867648 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:19.867687 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:30:19.869841 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:19.869862 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:19.870187 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:19.870553 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:19.872030 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:19.873416 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:19.874836 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:19.875689 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:19.875890 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:19.880769 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:19.880781 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:19.880816 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:19.880865 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:19.885086 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:19.885763 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:19.885772 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:19.885785 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:19.886002 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:19.886390 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:19.886396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:19.886405 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:19.887158 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:19.887457 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:19.887782 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:30:19.888252 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":374,"threads_started":1}
I20260806 22:30:19.889599 23064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:19.889698 23064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8833652d1d354a62a673df60c4d564e5, State: Initialized, Role: FOLLOWER
I20260806 22:30:19.890249 23064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } }
I20260806 22:30:19.890450 23064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:19.890551 23064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:19.890726 23064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:19.893390 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:30:19.893294 23064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } }
I20260806 22:30:19.893851 23064 leader_election.cc:302] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8833652d1d354a62a673df60c4d564e5; no voters:
I20260806 22:30:19.894469 23064 leader_election.cc:288] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:30:19.894575 23073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:19.896037 23073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8833652d1d354a62a673df60c4d564e5, State: Running, Role: LEADER
I20260806 22:30:19.896652 23064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:19.896629 23073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } }
I20260806 22:30:19.901363 23078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8833652d1d354a62a673df60c4d564e5. Latest consensus state: current_term: 1 leader_uuid: "8833652d1d354a62a673df60c4d564e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } } }
I20260806 22:30:19.901953 23078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:19.901093 23077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8833652d1d354a62a673df60c4d564e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8833652d1d354a62a673df60c4d564e5" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42029 } } }
I20260806 22:30:19.902318 23077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:19.908701 23089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:19.912907 23089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:19.921118 23089 catalog_manager.cc:1347] Generated new cluster ID: b35800d3fa274a8bbca3980fd15a56d6
I20260806 22:30:19.921247 23089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:19.933696 23089 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:30:19.935098 23089 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:19.941262 23089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8833652d1d354a62a673df60c4d564e5: Generated new TSK 0
I20260806 22:30:19.942123 23089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:30:19.955029 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 22961
I20260806 22:30:19.956238 23053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:20.046725 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 22961
2026-08-06T22:30:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2303 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:30:20 dist-test-slave-7vjv krb5kdc[23107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:30:20 dist-test-slave-7vjv krb5kdc[23107](info): set up 2 sockets
Aug 06 22:30:20 dist-test-slave-7vjv krb5kdc[23107](info): commencing operation
krb5kdc: starting...
W20260806 22:30:21.378700 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:30:21 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:30:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:30:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:30:21.683808 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:22.025733 23121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:22.026042 23121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:22.026212 23121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:22.033471 23121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:22.033581 23121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:22.033629 23121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:22.033735 23121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:22.033828 23121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:22.043776 23121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:22.045208 23121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:22.046535 23121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:22.058363 23132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:22.058331 23129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:22.058529 23127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:22.059016 23121 server_base.cc:1034] running on GCE node
I20260806 22:30:22.059849 23121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:22.061314 23121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:22.062549 23121 hybrid_clock.cc:648] HybridClock initialized: now 1786055422062499 us; error 56 us; skew 500 ppm
I20260806 22:30:22.062892 23121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:22 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055422, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:22.066212 23121 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:22.067826 23121 webserver.cc:460] Webserver started at http://127.1.8.126:45047/ using document root <none> and password file <none>
I20260806 22:30:22.068570 23121 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:22.068720 23121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:22.069028 23121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:22.072171 23121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "605f0e3dee16466f8c6b677fa9a7b598"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "5fb469fd6dafb9976c6c98761755d39e"
server_key_iv: "1a7b1b2e56e245b4de11c9c2d226f107"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.072971 23121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "605f0e3dee16466f8c6b677fa9a7b598"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "5fb469fd6dafb9976c6c98761755d39e"
server_key_iv: "1a7b1b2e56e245b4de11c9c2d226f107"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.078506 23121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:30:22.081857 23139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:22.083047 23121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 22:30:22.083245 23121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "605f0e3dee16466f8c6b677fa9a7b598"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "5fb469fd6dafb9976c6c98761755d39e"
server_key_iv: "1a7b1b2e56e245b4de11c9c2d226f107"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.083442 23121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:22.106215 23121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:22.110378 23121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:22.110713 23121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:22.129884 23121 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40241
I20260806 22:30:22.129933 23222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40241 every 8 connection(s)
I20260806 22:30:22.131275 23121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:22.132959 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 23121
I20260806 22:30:22.133193 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:30:22.133878 1057 external_mini_cluster.cc:1428] Setting key 759e43d7478593bd4646b25c3d7ff9b4
I20260806 22:30:22.135387 23223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:30:22 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:22.149576 23223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap starting.
I20260806 22:30:22.153126 23223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:22.154460 23223 log.cc:824] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:22.157507 23223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: No bootstrap required, opened a new log
I20260806 22:30:22.157809 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:22.135846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.1.8.126:40241)
0806 22:30:22.136624 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:22.136640 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:22.136690 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:30:22.138737 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:22.138764 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:22.139291 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:22.139758 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:22.141132 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.143226 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.144539 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.145549 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.145742 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:22.150295 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:22.150303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:22.150315 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:22.150351 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:22.153506 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:22.154386 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:22.154392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:22.154398 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:22.154605 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:22.155014 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:22.155021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:22.155026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:22.155586 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:22.155924 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:22.156366 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 22:30:22.156798 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":343,"threads_started":1}
I20260806 22:30:22.165127 23223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:22.165477 23223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:22.165586 23223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Initialized, Role: FOLLOWER
I20260806 22:30:22.166244 23223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:22.166610 23223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:22.166797 23223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:22.167023 23223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:22.169274 23223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:22.169721 23223 leader_election.cc:302] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 605f0e3dee16466f8c6b677fa9a7b598; no voters:
I20260806 22:30:22.170284 23223 leader_election.cc:288] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:30:22.170442 23231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:22.171124 23231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 LEADER]: Becoming Leader. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Running, Role: LEADER
I20260806 22:30:22.171654 23231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:22.172456 23223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:22.174414 23232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:22.175347 23232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:22.175043 23233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 605f0e3dee16466f8c6b677fa9a7b598. Latest consensus state: current_term: 1 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:22.180706 23233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:22.182173 23240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:22.187672 23240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:22.196082 23240 catalog_manager.cc:1347] Generated new cluster ID: ddb40c2e590545088ea2a08e3765d8e2
I20260806 22:30:22.196271 23240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:22.210929 23240 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:30:22.212764 23240 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:22.230247 23240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Generated new TSK 0
I20260806 22:30:22.231335 23240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:30:22.301716 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:22.648473 23261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:22.648739 23261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:22.648908 23261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:22.656239 23261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:22.656356 23261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:22.656538 23261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:22.668933 23261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:22.670205 23261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:22.671525 23261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:22.684983 23268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:22.685024 23271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:22.685050 23269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:22.685948 23261 server_base.cc:1034] running on GCE node
I20260806 22:30:22.686583 23261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:22.687587 23261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:22.688794 23261 hybrid_clock.cc:648] HybridClock initialized: now 1786055422688759 us; error 27 us; skew 500 ppm
I20260806 22:30:22.689110 23261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:22 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055422, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:22.692294 23261 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:22.693621 23261 webserver.cc:460] Webserver started at http://127.1.8.65:35871/ using document root <none> and password file <none>
I20260806 22:30:22.694248 23261 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:22.694345 23261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:22.694604 23261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:22.697472 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "03ac967012ef40f6bce314fd4fd41f6a"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "b718e940b6e46511b62573d6baab8277"
server_key_iv: "70991f4a6d070d8ac57336d664836078"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.698244 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "03ac967012ef40f6bce314fd4fd41f6a"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "b718e940b6e46511b62573d6baab8277"
server_key_iv: "70991f4a6d070d8ac57336d664836078"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.703233 23261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:30:22.706456 23284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:22.707724 23261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:30:22.707908 23261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "03ac967012ef40f6bce314fd4fd41f6a"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "b718e940b6e46511b62573d6baab8277"
server_key_iv: "70991f4a6d070d8ac57336d664836078"
server_key_version: "encryptionkey@0"
I20260806 22:30:22.708083 23261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:22.732481 23261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:22.737154 23261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:22.737416 23261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:22.738373 23261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:22.739698 23261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:22.739770 23261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:22.739852 23261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:22.739897 23261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:22.773604 23261 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44769
I20260806 22:30:22.773694 23460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44769 every 8 connection(s)
I20260806 22:30:22.774916 23261 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:22.779422 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 23261
I20260806 22:30:22.779692 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:30:22.780589 1057 external_mini_cluster.cc:1428] Setting key 9d32c36a9cce4f3b9c0f59fc9081a85d
Aug 06 22:30:22 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055422, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:22.798645 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:22.777411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:57265 (local address 127.1.8.126:40241)
0806 22:30:22.777850 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:22.777855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:22.778916 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 22:30:22.781610 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:22.781616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:22.781721 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:22.781815 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:22.783466 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.784654 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.786141 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.786901 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.787072 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:22.791689 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:22.791697 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:22.791704 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:22.791748 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:22.795147 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:22.796195 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:22.796200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:22.796202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:22.796329 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:22.796871 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:22.796874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:22.796876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:22.797142 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:22.797251 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:22.798198 (+ 947us) server_negotiation.cc:299] Negotiation successful
0806 22:30:22.798376 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":217,"threads_started":1}
I20260806 22:30:22.800262 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:22.778079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:57265)
0806 22:30:22.778763 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:22.778799 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:22.781350 (+ 2551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:22.782098 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:22.782109 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:22.782737 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:22.783283 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:22.783300 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.784796 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.784800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:22.785991 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:22.786009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:22.787152 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:22.787160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:22.787457 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:22.788368 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:22.788400 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:22.791459 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:22.795303 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:22.795315 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:22.795335 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:22.796002 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:22.796456 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:22.796462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:22.796469 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:22.796750 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:22.797297 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:22.797308 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:22.797789 (+ 481us) client_negotiation.cc:769] Sending connection context
0806 22:30:22.798106 (+ 317us) client_negotiation.cc:240] Negotiation successful
0806 22:30:22.798716 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":275,"threads_started":1}
I20260806 22:30:22.801951 23464 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:22.802291 23464 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:22.803018 23464 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:22.805454 23167 ts_manager.cc:194] Registered new tserver with Master: 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769)
I20260806 22:30:22.807655 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:57265
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:30:22.851701 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:23.189458 23471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:23.189738 23471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:23.189834 23471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:23.196952 23471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:23.197068 23471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:23.197181 23471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:23.206763 23471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:23.208010 23471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:23.209337 23471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:23.222741 23477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:23.222718 23486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:23.222885 23480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:23.224016 23471 server_base.cc:1034] running on GCE node
I20260806 22:30:23.224655 23471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:23.225530 23471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:23.226727 23471 hybrid_clock.cc:648] HybridClock initialized: now 1786055423226661 us; error 51 us; skew 500 ppm
I20260806 22:30:23.227049 23471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:23 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:23.230187 23471 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:23.231729 23471 webserver.cc:460] Webserver started at http://127.1.8.66:45431/ using document root <none> and password file <none>
I20260806 22:30:23.232549 23471 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:23.232695 23471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:23.233033 23471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:23.235965 23471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "e511ac4bd71f45828f040cd0a2036970"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "1d3b3dfebe092d60623a8e1a9a010e84"
server_key_iv: "68b0b0a901fd24780c8e4b1b75d19941"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.236837 23471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "e511ac4bd71f45828f040cd0a2036970"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "1d3b3dfebe092d60623a8e1a9a010e84"
server_key_iv: "68b0b0a901fd24780c8e4b1b75d19941"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.241936 23471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:30:23.245397 23497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:23.246711 23471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:30:23.246891 23471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "e511ac4bd71f45828f040cd0a2036970"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "1d3b3dfebe092d60623a8e1a9a010e84"
server_key_iv: "68b0b0a901fd24780c8e4b1b75d19941"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.247092 23471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:23.289215 23471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:23.293164 23471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:23.293467 23471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:23.294463 23471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:23.295830 23471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:23.295936 23471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:23.296087 23471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:23.296190 23471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 22:30:23.330125 23471 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:46351
I20260806 22:30:23.330214 23669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:46351 every 8 connection(s)
I20260806 22:30:23.331529 23471 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:23.341756 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 23471
I20260806 22:30:23.341969 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:30:23.342576 1057 external_mini_cluster.cc:1428] Setting key 371117d49423074a4810a430b02b24ae
Aug 06 22:30:23 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:23.352445 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.334153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:48323 (local address 127.1.8.126:40241)
0806 22:30:23.334637 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.334642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.335574 (+ 932us) server_negotiation.cc:407] Connection header received
0806 22:30:23.337440 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.337447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.337543 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.337680 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:23.339152 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.339897 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.341565 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.342118 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.342222 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.346056 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:23.346064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:23.346072 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:23.346139 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:23.349294 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.350060 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.350065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.350067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.350205 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.350689 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.350693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.350695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.351005 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:23.351172 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.351818 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.352045 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":265,"threads_started":1}
I20260806 22:30:23.353390 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.334623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:48323)
0806 22:30:23.335400 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:23.335445 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:23.337161 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:23.337920 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:23.337927 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:23.338403 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:23.338953 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.338969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.340066 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.340071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.341317 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.341330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.342349 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.342356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.342663 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.343373 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:23.343395 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:23.345802 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:23.349444 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:23.349453 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:23.349469 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:23.349906 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:23.350321 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:23.350327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:23.350331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:23.350521 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:23.351146 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:23.351154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:23.351452 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 22:30:23.351708 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 22:30:23.352293 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":388,"threads_started":1}
I20260806 22:30:23.354852 23670 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:23.355183 23670 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:23.356029 23670 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:23.357596 23167 ts_manager.cc:194] Registered new tserver with Master: e511ac4bd71f45828f040cd0a2036970 (127.1.8.66:46351)
I20260806 22:30:23.358367 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:48323
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:30:23.407519 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:23.741585 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:23.741847 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:23.741940 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:23.749035 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:23.749142 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:23.749254 23679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:23.758919 23679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:23.760139 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:23.761399 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:23.774935 23686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:23.775004 23687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:23.775215 23689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:23.776757 23679 server_base.cc:1034] running on GCE node
I20260806 22:30:23.777511 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:23.778614 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:23.779848 23679 hybrid_clock.cc:648] HybridClock initialized: now 1786055423779803 us; error 58 us; skew 500 ppm
I20260806 22:30:23.780207 23679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:23 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:23.783278 23679 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:23.784704 23679 webserver.cc:460] Webserver started at http://127.1.8.67:43155/ using document root <none> and password file <none>
I20260806 22:30:23.785399 23679 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:23.785530 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:23.785843 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:23.788936 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "5665b1f494ea4d7199b4f20eedc66f56"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "fb7b72432594567b99b78b7f5cf4010f"
server_key_iv: "5bd3004798613857aa2bab70e92264e7"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.789697 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "5665b1f494ea4d7199b4f20eedc66f56"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "fb7b72432594567b99b78b7f5cf4010f"
server_key_iv: "5bd3004798613857aa2bab70e92264e7"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.794704 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:30:23.798054 23701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:23.799352 23679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:30:23.799520 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5665b1f494ea4d7199b4f20eedc66f56"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "fb7b72432594567b99b78b7f5cf4010f"
server_key_iv: "5bd3004798613857aa2bab70e92264e7"
server_key_version: "encryptionkey@0"
I20260806 22:30:23.799734 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:23.811035 23464 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:23.816706 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:23.820515 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:23.820801 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:23.822417 23679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:23.823793 23679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:23.823897 23679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:23.824026 23679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:23.824126 23679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:23.858414 23679 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:37041
I20260806 22:30:23.858505 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:37041 every 8 connection(s)
I20260806 22:30:23.859702 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:23.865031 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 23679
I20260806 22:30:23.865243 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:30:23.865887 1057 external_mini_cluster.cc:1428] Setting key d15158690fbe7c51b39da15576de2b25
Aug 06 22:30:23 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:23.881801 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.862432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:33097 (local address 127.1.8.126:40241)
0806 22:30:23.862905 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.862910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.863918 (+ 1008us) server_negotiation.cc:407] Connection header received
0806 22:30:23.866655 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.866662 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.866763 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.866915 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:23.868841 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.869694 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.871447 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.871875 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.872043 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.875640 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:23.875645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:23.875649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:23.875682 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:23.878545 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.879216 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.879220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.879223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.879356 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.879892 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.879896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.879898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.880318 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:23.880458 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.881334 (+ 876us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.881537 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":220,"threads_started":1}
I20260806 22:30:23.882992 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.862976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:33097)
0806 22:30:23.863706 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:23.863743 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:23.866276 (+ 2533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:23.867198 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:23.867207 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:23.867980 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:23.868629 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.868662 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.869878 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.869884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.871230 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.871243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.872182 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.872191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.872434 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.873122 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:23.873166 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:23.875416 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:23.878685 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:23.878691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:23.878701 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:23.879100 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:23.879472 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:23.879476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:23.879479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:23.879730 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:23.880470 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:23.880477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:23.880894 (+ 417us) client_negotiation.cc:769] Sending connection context
0806 22:30:23.881189 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 22:30:23.881993 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":289,"threads_started":1}
I20260806 22:30:23.884317 23882 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:23.884568 23882 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:23.885226 23882 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:23.886585 23167 ts_manager.cc:194] Registered new tserver with Master: 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041)
I20260806 22:30:23.887333 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:33097
I20260806 22:30:23.893384 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:30:23.909828 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.897868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.1.8.126:40241)
0806 22:30:23.898097 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.898105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.898466 (+ 361us) server_negotiation.cc:407] Connection header received
0806 22:30:23.898587 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.898591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.898648 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.898725 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:23.900139 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.900827 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.902126 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.902485 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.902572 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.905000 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:23.905005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:23.905008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:23.905036 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:23.907618 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.908237 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.908240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.908243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.908356 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:23.908668 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:23.908671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:23.908673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:23.908965 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:23.909156 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.909346 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.909505 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:30:23.914134 23167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:30:23.921747 23167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:30:23.954939 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.935693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.1.8.67:37041)
0806 22:30:23.946480 (+ 10787us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.946489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.946523 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:30:23.946622 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.946629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.946823 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.947906 (+ 1083us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:23.948285 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.950056 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.952740 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.953710 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.953857 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.954107 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.954413 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.954585 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10614,"thread_start_us":10486,"threads_started":1}
I20260806 22:30:23.955637 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.934968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37041 (local address 127.0.0.1:40262)
0806 22:30:23.944986 (+ 10018us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:23.945014 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:23.945196 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:23.947074 (+ 1878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:23.947082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:23.947125 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:23.947690 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.947711 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.950206 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.950211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.952586 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.952598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.953900 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.953906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.954161 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.954213 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:30:23.954340 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 22:30:23.954525 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9821,"mutex_wait_us":61,"spinlock_wait_cycles":1280,"thread_start_us":10342,"threads_started":1}
I20260806 22:30:23.956497 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.934193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46828)
0806 22:30:23.934939 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:23.934980 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:23.935165 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:23.947073 (+ 11908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:23.947082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:23.947132 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:23.948409 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.948423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.950337 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.950342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.953587 (+ 3245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.953603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.955629 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.955636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.955820 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.955851 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:30:23.955938 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:30:23.956082 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":733,"spinlock_wait_cycles":3840,"thread_start_us":326,"threads_started":1}
I20260806 22:30:23.957160 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.934512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.1.8.65:44769)
0806 22:30:23.946068 (+ 11556us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.946080 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.946117 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:30:23.946235 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.946244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.946561 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.946772 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:23.948652 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.950150 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.953770 (+ 3620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.955156 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.955266 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.955514 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.956718 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.956881 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11352,"thread_start_us":11204,"threads_started":1}
I20260806 22:30:23.957823 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.935354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:46351 (local address 127.0.0.1:43640)
0806 22:30:23.947666 (+ 12312us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:23.947685 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:23.947798 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:23.949740 (+ 1942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:23.949745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:23.949772 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:23.950155 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.950168 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.952291 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.952299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.955069 (+ 2770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:23.955082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.957032 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.957038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:23.957211 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.957240 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:30:23.957331 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:30:23.957447 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12163,"mutex_wait_us":1,"thread_start_us":12017,"threads_started":1}
I20260806 22:30:23.958528 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:23.935727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.1.8.66:46351)
0806 22:30:23.949292 (+ 13565us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:23.949302 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:23.949331 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:30:23.949402 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:23.949407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:23.949551 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:23.949748 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:23.950344 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.952127 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.955256 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:23.956528 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:23.956684 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:23.956877 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:23.958060 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0806 22:30:23.958234 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13409,"thread_start_us":13293,"threads_started":1}
I20260806 22:30:23.960044 23781 tablet_service.cc:1467] Processing CreateTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 (DEFAULT_TABLE table=test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:23.960428 23357 tablet_service.cc:1467] Processing CreateTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 (DEFAULT_TABLE table=test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:23.961506 23781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91171400131a4d758bb8e50de1b9b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:23.961886 23357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91171400131a4d758bb8e50de1b9b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:23.961967 23565 tablet_service.cc:1467] Processing CreateTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 (DEFAULT_TABLE table=test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:23.963264 23565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91171400131a4d758bb8e50de1b9b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:23.975191 23907 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Bootstrap starting.
I20260806 22:30:23.976397 23908 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Bootstrap starting.
I20260806 22:30:23.978094 23909 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Bootstrap starting.
I20260806 22:30:23.979261 23907 tablet_bootstrap.cc:654] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:23.979974 23908 tablet_bootstrap.cc:654] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:23.981338 23907 log.cc:824] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:23.981338 23908 log.cc:824] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:23.981386 23909 tablet_bootstrap.cc:654] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:23.982591 23909 log.cc:824] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:23.984695 23907 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: No bootstrap required, opened a new log
I20260806 22:30:23.984987 23907 ts_tablet_manager.cc:1397] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 22:30:23.985283 23908 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: No bootstrap required, opened a new log
I20260806 22:30:23.985569 23908 ts_tablet_manager.cc:1397] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260806 22:30:23.985698 23909 tablet_bootstrap.cc:492] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: No bootstrap required, opened a new log
I20260806 22:30:23.985958 23909 ts_tablet_manager.cc:1397] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 22:30:23.993321 23907 raft_consensus.cc:348] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:23.993706 23907 raft_consensus.cc:374] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:23.993811 23907 raft_consensus.cc:729] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5665b1f494ea4d7199b4f20eedc66f56, State: Initialized, Role: FOLLOWER
I20260806 22:30:23.994488 23907 consensus_queue.cc:260] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:23.995644 23882 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:23.996349 23908 raft_consensus.cc:348] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:23.996788 23908 raft_consensus.cc:374] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:23.996886 23908 raft_consensus.cc:729] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ac967012ef40f6bce314fd4fd41f6a, State: Initialized, Role: FOLLOWER
I20260806 22:30:23.996681 23909 raft_consensus.cc:348] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:23.997138 23909 raft_consensus.cc:374] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:23.997267 23909 raft_consensus.cc:729] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e511ac4bd71f45828f040cd0a2036970, State: Initialized, Role: FOLLOWER
I20260806 22:30:23.997905 23909 consensus_queue.cc:260] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:23.998147 23907 ts_tablet_manager.cc:1428] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Time spent starting tablet: real 0.013s user 0.007s sys 0.004s
I20260806 22:30:23.998992 23670 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:23.999701 23909 ts_tablet_manager.cc:1428] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260806 22:30:24.000191 23908 consensus_queue.cc:260] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:24.001971 23908 ts_tablet_manager.cc:1428] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260806 22:30:24.027412 23466 tablet.cc:2369] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:30:24.083777 23672 tablet.cc:2369] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:30:24.111503 23883 tablet.cc:2369] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:24.147593 23917 raft_consensus.cc:482] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:30:24.147950 23917 raft_consensus.cc:504] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:24.149909 23917 leader_election.cc:288] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769), 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041)
I20260806 22:30:24.159322 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.150527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:50225 (local address 127.1.8.67:37041)
0806 22:30:24.150823 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.150832 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.151271 (+ 439us) server_negotiation.cc:407] Connection header received
0806 22:30:24.151517 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.151521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.151634 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.151733 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:24.152534 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.153731 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.156810 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.158059 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.158224 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.158331 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.158951 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.159082 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 22:30:24.159327 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.150323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:50613 (local address 127.1.8.65:44769)
0806 22:30:24.150588 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.150598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.151576 (+ 978us) server_negotiation.cc:407] Connection header received
0806 22:30:24.151687 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.151694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.151770 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.151861 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:24.152647 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.153729 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.156810 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.158050 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.158223 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.158331 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.158951 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.159090 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:30:24.159574 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.150233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37041 (local address 127.1.8.66:50225)
0806 22:30:24.151159 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:24.151176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:24.151362 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:24.151757 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:24.151760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:24.151779 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:24.152369 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:24.152386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.153873 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.153878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:24.156642 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:24.156656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.158523 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.158529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:24.158743 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.158819 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 22:30:24.158911 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:30:24.159055 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":257,"thread_start_us":913,"threads_started":1}
I20260806 22:30:24.159615 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.150263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.1.8.66:50613)
0806 22:30:24.151463 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:24.151482 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:24.151586 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:24.151902 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:24.151907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:24.151944 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:24.152466 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:24.152481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.153874 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.153878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:24.156641 (+ 2763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:24.156656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.158516 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.158523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:24.158742 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.158809 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 22:30:24.158911 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:30:24.159055 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"mutex_wait_us":191,"thread_start_us":531,"threads_started":1}
I20260806 22:30:24.161098 23813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91171400131a4d758bb8e50de1b9b4e6" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665b1f494ea4d7199b4f20eedc66f56" is_pre_election: true
I20260806 22:30:24.161088 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91171400131a4d758bb8e50de1b9b4e6" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ac967012ef40f6bce314fd4fd41f6a" is_pre_election: true
I20260806 22:30:24.161686 23813 raft_consensus.cc:2454] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 0.
I20260806 22:30:24.161686 23391 raft_consensus.cc:2454] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 0.
I20260806 22:30:24.162429 23503 leader_election.cc:302] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03ac967012ef40f6bce314fd4fd41f6a, e511ac4bd71f45828f040cd0a2036970; no voters:
I20260806 22:30:24.162881 23917 raft_consensus.cc:2781] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:30:24.163043 23917 raft_consensus.cc:482] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:30:24.163166 23917 raft_consensus.cc:3037] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:24.165454 23917 raft_consensus.cc:504] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:24.166038 23917 leader_election.cc:288] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 election: Requested vote from peers 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769), 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041)
I20260806 22:30:24.166496 23813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91171400131a4d758bb8e50de1b9b4e6" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665b1f494ea4d7199b4f20eedc66f56"
I20260806 22:30:24.166499 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91171400131a4d758bb8e50de1b9b4e6" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ac967012ef40f6bce314fd4fd41f6a"
I20260806 22:30:24.166672 23813 raft_consensus.cc:3037] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:24.166704 23391 raft_consensus.cc:3037] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:24.168916 23813 raft_consensus.cc:2454] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 1.
I20260806 22:30:24.169212 23391 raft_consensus.cc:2454] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 1.
I20260806 22:30:24.169339 23501 leader_election.cc:302] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5665b1f494ea4d7199b4f20eedc66f56, e511ac4bd71f45828f040cd0a2036970; no voters:
I20260806 22:30:24.169641 23917 raft_consensus.cc:2781] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:24.170125 23917 raft_consensus.cc:686] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 1 LEADER]: Becoming Leader. State: Replica: e511ac4bd71f45828f040cd0a2036970, State: Running, Role: LEADER
I20260806 22:30:24.170591 23917 consensus_queue.cc:237] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } }
I20260806 22:30:24.175437 23166 catalog_manager.cc:5495] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 reported cstate change: term changed from 0 to 1, leader changed from <none> to e511ac4bd71f45828f040cd0a2036970 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "e511ac4bd71f45828f040cd0a2036970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } health_report { overall_health: UNKNOWN } } }
I20260806 22:30:24.205897 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.200600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.1.8.66:46351)
0806 22:30:24.200771 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.200776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.200796 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:30:24.200984 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.200988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.201065 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.201193 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:30:24.201512 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.202224 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.203703 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.204220 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.204321 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.204827 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:30:24.205338 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:30:24.205481 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.205592 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.205663 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:30:24.214845 23813 raft_consensus.cc:1261] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Refusing update from remote peer e511ac4bd71f45828f040cd0a2036970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:24.214847 23391 raft_consensus.cc:1261] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Refusing update from remote peer e511ac4bd71f45828f040cd0a2036970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:24.215813 23922 consensus_queue.cc:1035] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:24.216228 23917 consensus_queue.cc:1035] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:24.227780 23929 mvcc.cc:205] Tried to move back new op lower bound from 7315683017572966400 to 7315683017406840832. Current Snapshot: MvccSnapshot[applied={T|T < 7315683017572966400}]
I20260806 22:30:24.233161 23931 mvcc.cc:205] Tried to move back new op lower bound from 7315683017572966400 to 7315683017406840832. Current Snapshot: MvccSnapshot[applied={T|T < 7315683017572966400}]
I20260806 22:30:24.244556 23166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:24.244933 23166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:24.251349 23166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Sending DeleteTablet for 3 replicas of tablet 91171400131a4d758bb8e50de1b9b4e6
I20260806 22:30:24.253059 23565 tablet_service.cc:1514] Processing DeleteTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:24 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:43640
I20260806 22:30:24.253386 23357 tablet_service.cc:1514] Processing DeleteTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:24 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:46828
I20260806 22:30:24.253957 23944 tablet_replica.cc:330] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: stopping tablet replica
I20260806 22:30:24.254638 23781 tablet_service.cc:1514] Processing DeleteTablet for tablet 91171400131a4d758bb8e50de1b9b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:24 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40262
I20260806 22:30:24.254773 23944 raft_consensus.cc:2229] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:30:24.254878 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 23261
I20260806 22:30:24.255496 23944 raft_consensus.cc:2258] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:24.258322 23944 ts_tablet_manager.cc:1905] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:24.262097 23946 tablet_replica.cc:330] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: stopping tablet replica
I20260806 22:30:24.262768 23946 raft_consensus.cc:2229] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:24.263761 23946 raft_consensus.cc:2258] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:24.267125 23946 ts_tablet_manager.cc:1905] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:24.267158 23944 ts_tablet_manager.cc:1918] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:24.267338 23944 log.cc:1196] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/91171400131a4d758bb8e50de1b9b4e6
I20260806 22:30:24.267683 23947 tablet_replica.cc:330] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: stopping tablet replica
I20260806 22:30:24.268136 23944 ts_tablet_manager.cc:1939] T 91171400131a4d758bb8e50de1b9b4e6 P e511ac4bd71f45828f040cd0a2036970: Deleting consensus metadata
I20260806 22:30:24.268201 23947 raft_consensus.cc:2229] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:24.268847 23947 raft_consensus.cc:2258] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:24.270340 23144 catalog_manager.cc:4841] TS e511ac4bd71f45828f040cd0a2036970 (127.1.8.66:46351): tablet 91171400131a4d758bb8e50de1b9b4e6 (table test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]) successfully deleted
I20260806 22:30:24.271045 23947 ts_tablet_manager.cc:1905] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:24.276324 23946 ts_tablet_manager.cc:1918] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:24.276530 23946 log.cc:1196] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/91171400131a4d758bb8e50de1b9b4e6
I20260806 22:30:24.277135 23946 ts_tablet_manager.cc:1939] T 91171400131a4d758bb8e50de1b9b4e6 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting consensus metadata
I20260806 22:30:24.278733 23947 ts_tablet_manager.cc:1918] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:24.278946 23947 log.cc:1196] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/91171400131a4d758bb8e50de1b9b4e6
I20260806 22:30:24.279174 23144 catalog_manager.cc:4841] TS 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769): tablet 91171400131a4d758bb8e50de1b9b4e6 (table test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]) successfully deleted
I20260806 22:30:24.279668 23947 ts_tablet_manager.cc:1939] T 91171400131a4d758bb8e50de1b9b4e6 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting consensus metadata
I20260806 22:30:24.281145 23141 catalog_manager.cc:4841] TS 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041): tablet 91171400131a4d758bb8e50de1b9b4e6 (table test-table [id=5bb5d3eead1145faa4dbb19b09d0c5ae]) successfully deleted
Aug 06 22:30:24 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:30:24.291365 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.261675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.1.8.65:44769)
0806 22:30:24.261861 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.261867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.272066 (+ 10199us) server_negotiation.cc:407] Connection header received
0806 22:30:24.272338 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.272346 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.272433 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.272541 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:24.274660 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.275871 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.278208 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.278921 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.279024 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.284444 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:24.284459 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:24.284477 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:24.284529 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:24.288433 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.289136 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.289146 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.289150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.289319 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.289694 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.289699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.289703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.290185 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:24.290499 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.290720 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.290953 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:30:24.292148 23455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:24.421836 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 23261
I20260806 22:30:24.451644 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 23471
Aug 06 22:30:24 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:30:24.473278 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.452570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.1.8.66:46351)
0806 22:30:24.452752 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.452760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.452787 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:30:24.453028 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.453034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.453112 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.453221 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:24.454517 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.455387 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.456856 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.457548 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.457683 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.466277 (+ 8594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:24.466290 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:24.466314 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:24.466372 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:24.470286 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.470911 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.470925 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.470929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.471098 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.471442 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.471448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.471451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.471896 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:24.472267 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.472650 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.472875 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:30:24.474032 23662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:24.593914 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 23471
I20260806 22:30:24.619700 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 23679
Aug 06 22:30:24 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:30:24.634143 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:24.620447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.1.8.67:37041)
0806 22:30:24.620623 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:24.620627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:24.620645 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:30:24.620763 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:24.620767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:24.620829 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:24.620916 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:24.622069 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.622777 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.623926 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:24.624480 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:24.624573 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:24.628802 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:24.628815 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:24.628832 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:24.628869 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:24.631642 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.632446 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.632453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.632456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.632614 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:24.632931 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:24.632934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:24.632936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:24.633263 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:24.633480 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:24.633720 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 22:30:24.633878 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:30:24.634771 23876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:24.723956 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 23679
I20260806 22:30:24.747629 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 23121
I20260806 22:30:24.748934 23216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:24.861048 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 23121
I20260806 22:30:24.891170 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--webserver_interface=127.1.8.126
--webserver_port=45047
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:25.234575 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:25.234858 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:25.234947 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:25.241981 23956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:25.242093 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:25.242147 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:25.242198 23956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:25.242249 23956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:25.251729 23956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=45047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:25.253014 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:25.254458 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:25.281457 23965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:25.281457 23966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:25.281755 23969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:25.282544 23956 server_base.cc:1034] running on GCE node
I20260806 22:30:25.283442 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:25.285161 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:25.286432 23956 hybrid_clock.cc:648] HybridClock initialized: now 1786055425286364 us; error 50 us; skew 500 ppm
I20260806 22:30:25.286891 23956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:25 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055425, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:25.293502 23956 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:25.295367 23956 webserver.cc:460] Webserver started at http://127.1.8.126:45047/ using document root <none> and password file <none>
I20260806 22:30:25.296192 23956 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:25.296319 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:25.302327 23956 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 22:30:25.305128 23979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:25.306479 23956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:30:25.306679 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "605f0e3dee16466f8c6b677fa9a7b598"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "5fb469fd6dafb9976c6c98761755d39e"
server_key_iv: "1a7b1b2e56e245b4de11c9c2d226f107"
server_key_version: "encryptionkey@0"
I20260806 22:30:25.307773 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:25.326500 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:25.331225 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:25.331516 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:25.351156 23956 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40241
I20260806 22:30:25.351181 24051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40241 every 8 connection(s)
I20260806 22:30:25.352739 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:25.359717 24052 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:30:25.362874 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 23956
I20260806 22:30:25.363449 24052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap starting.
I20260806 22:30:25.364001 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:44769
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=35871
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:25.387324 24052 log.cc:824] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:25.401850 24052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:30:25.402416 24052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap complete.
I20260806 22:30:25.411933 24052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:25.412923 24052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Initialized, Role: FOLLOWER
I20260806 22:30:25.413591 24052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:25.413844 24052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:25.413971 24052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:25.414151 24052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:30:25.417939 24052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:25.418514 24052 leader_election.cc:302] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 605f0e3dee16466f8c6b677fa9a7b598; no voters:
I20260806 22:30:25.419170 24052 leader_election.cc:288] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:30:25.419297 24057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:30:25.420620 24057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 LEADER]: Becoming Leader. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Running, Role: LEADER
I20260806 22:30:25.421227 24057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:25.421576 24052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:25.424194 24058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:25.424240 24059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 605f0e3dee16466f8c6b677fa9a7b598. Latest consensus state: current_term: 2 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:25.424603 24058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:25.424647 24059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:25.437637 24068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:25.448195 24068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:25.450286 24068 catalog_manager.cc:1259] Loaded cluster ID: ddb40c2e590545088ea2a08e3765d8e2
I20260806 22:30:25.450471 24068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:25.454716 24068 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:30:25.481884 24068 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:30:25.508600 24068 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:25.510495 24068 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Loaded TSK: 0
I20260806 22:30:25.512295 24068 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:30:25.743230 24055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:25.743579 24055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:25.743762 24055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:25.750941 24055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:25.751102 24055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:25.751266 24055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:25.760808 24055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:44769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=35871
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:25.762069 24055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:25.763386 24055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:25.779042 24095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:25.778955 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:25.779702 24099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:25.780131 24055 server_base.cc:1034] running on GCE node
I20260806 22:30:25.780974 24055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:25.782146 24055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:25.783403 24055 hybrid_clock.cc:648] HybridClock initialized: now 1786055425783343 us; error 61 us; skew 500 ppm
I20260806 22:30:25.783785 24055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:25 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055425, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:25.787787 24055 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:25.789428 24055 webserver.cc:460] Webserver started at http://127.1.8.65:35871/ using document root <none> and password file <none>
I20260806 22:30:25.790179 24055 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:25.790334 24055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:25.796397 24055 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:30:25.799285 24112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:25.800786 24055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:30:25.800974 24055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "03ac967012ef40f6bce314fd4fd41f6a"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "b718e940b6e46511b62573d6baab8277"
server_key_iv: "70991f4a6d070d8ac57336d664836078"
server_key_version: "encryptionkey@0"
I20260806 22:30:25.801975 24055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:25.835450 24055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:25.840009 24055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:25.840322 24055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:25.841344 24055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:25.842715 24055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:25.842850 24055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:25.842993 24055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:25.843091 24055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:25.878376 24055 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44769
I20260806 22:30:25.878464 24282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44769 every 8 connection(s)
I20260806 22:30:25.879968 24055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:25.886426 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 24055
I20260806 22:30:25.887079 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:46351
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=45431
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:25 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055425, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:25.905866 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:25.883025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:33601 (local address 127.1.8.126:40241)
0806 22:30:25.883759 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:25.883774 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:25.884173 (+ 399us) server_negotiation.cc:407] Connection header received
0806 22:30:25.886520 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:25.886532 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:25.886900 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:25.887275 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:25.888975 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:25.890261 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:25.891943 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:25.892680 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:25.892916 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:25.897782 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:25.897794 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:25.897815 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:25.897867 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:25.901254 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:25.902208 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:25.902215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:25.902220 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:25.902377 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:25.902876 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:25.902879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:25.902881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:25.903324 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:25.903477 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:25.904428 (+ 951us) server_negotiation.cc:299] Negotiation successful
0806 22:30:25.904761 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":309,"threads_started":1}
I20260806 22:30:25.906157 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:25.883048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:33601)
0806 22:30:25.883940 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:25.883988 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:25.885912 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:25.887434 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:25.887447 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:25.888144 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:25.888714 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:25.888731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:25.890472 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:25.890476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:25.891776 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:25.891791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:25.892978 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:25.892987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:25.893265 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:25.893999 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:25.894023 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:25.897474 (+ 3451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:25.901397 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:25.901409 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:25.901426 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:25.902072 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:25.902491 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:25.902499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:25.902522 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:25.902762 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:25.903482 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:25.903494 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:25.903932 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 22:30:25.904258 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 22:30:25.904820 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":399,"threads_started":1}
I20260806 22:30:25.909126 24284 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:25.909469 24284 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:25.910403 24284 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:25.912766 24002 ts_manager.cc:194] Registered new tserver with Master: 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769)
I20260806 22:30:25.914296 24002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:33601
W20260806 22:30:26.236653 24289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:26.236995 24289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:26.237191 24289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:26.244403 24289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:26.244552 24289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:26.244745 24289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:26.254333 24289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:46351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=45431
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:26.255673 24289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:26.257078 24289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:26.270074 24296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:26.270164 24297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:26.270346 24300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:26.271266 24289 server_base.cc:1034] running on GCE node
I20260806 22:30:26.271924 24289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:26.272893 24289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:26.274111 24289 hybrid_clock.cc:648] HybridClock initialized: now 1786055426274035 us; error 60 us; skew 500 ppm
I20260806 22:30:26.274425 24289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:26 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:26.277812 24289 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:26.279292 24289 webserver.cc:460] Webserver started at http://127.1.8.66:45431/ using document root <none> and password file <none>
I20260806 22:30:26.280174 24289 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:26.280335 24289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:26.285753 24289 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:30:26.288445 24313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.289772 24289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:30:26.289958 24289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "e511ac4bd71f45828f040cd0a2036970"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "1d3b3dfebe092d60623a8e1a9a010e84"
server_key_iv: "68b0b0a901fd24780c8e4b1b75d19941"
server_key_version: "encryptionkey@0"
I20260806 22:30:26.290934 24289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:26.307543 24289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:26.311822 24289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:26.312151 24289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:26.313254 24289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:26.314533 24289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:26.314666 24289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.314793 24289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:26.314894 24289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.362334 24289 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:46351
I20260806 22:30:26.362375 24479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:46351 every 8 connection(s)
I20260806 22:30:26.363749 24289 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:26.364822 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 24289
I20260806 22:30:26.365537 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:37041
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=43155
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:30:26 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:26.387176 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.368838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:41465 (local address 127.1.8.126:40241)
0806 22:30:26.369072 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.369079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.370338 (+ 1259us) server_negotiation.cc:407] Connection header received
0806 22:30:26.372414 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.372420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.372527 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.372649 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:26.374112 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.374830 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.376574 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.377036 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.377180 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.381225 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:26.381231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:26.381236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:26.381276 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:26.383954 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.384753 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.384759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.384763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.384907 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.385408 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.385419 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.385427 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.385693 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:26.385855 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.386548 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.386795 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:30:26.388193 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.369362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:41465)
0806 22:30:26.370183 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:26.370228 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:26.372080 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:26.372887 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:26.372895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:26.373384 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:26.373927 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.373943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.374994 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.374999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.376421 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.376438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.377283 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.377290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.377529 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.378212 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:26.378244 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:26.381016 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:26.384155 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:26.384166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:26.384182 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:26.384619 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:26.385024 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:26.385029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:26.385033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:26.385287 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:26.385828 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:26.385837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:26.386176 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 22:30:26.386432 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 22:30:26.386989 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":506,"threads_started":1}
I20260806 22:30:26.389705 24480 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:26.390002 24480 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:26.390733 24480 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:26.392309 24002 ts_manager.cc:194] Registered new tserver with Master: e511ac4bd71f45828f040cd0a2036970 (127.1.8.66:46351)
I20260806 22:30:26.393646 24002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:41465
W20260806 22:30:26.706355 24484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:26.706663 24484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:26.706847 24484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:26.714022 24484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:26.714146 24484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:26.714337 24484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:26.723934 24484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:37041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=43155
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:26.725183 24484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:26.726485 24484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:26.739168 24498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:26.739264 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:26.739220 24496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:26.740176 24484 server_base.cc:1034] running on GCE node
I20260806 22:30:26.740783 24484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:26.741664 24484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:26.742889 24484 hybrid_clock.cc:648] HybridClock initialized: now 1786055426742849 us; error 62 us; skew 500 ppm
I20260806 22:30:26.743197 24484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:26 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:26.746284 24484 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:26.747612 24484 webserver.cc:460] Webserver started at http://127.1.8.67:43155/ using document root <none> and password file <none>
I20260806 22:30:26.748306 24484 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:26.748427 24484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:26.753398 24484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:30:26.755959 24511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.757272 24484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:30:26.757436 24484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5665b1f494ea4d7199b4f20eedc66f56"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "fb7b72432594567b99b78b7f5cf4010f"
server_key_iv: "5bd3004798613857aa2bab70e92264e7"
server_key_version: "encryptionkey@0"
I20260806 22:30:26.758351 24484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:26.772579 24484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:26.776286 24484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:26.776543 24484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:26.777529 24484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:26.778750 24484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:26.778834 24484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.778973 24484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:26.779062 24484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:26.812312 24484 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:37041
I20260806 22:30:26.812398 24668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:37041 every 8 connection(s)
I20260806 22:30:26.813707 24484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:26.817777 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 24484
Aug 06 22:30:26 dist-test-slave-7vjv krb5kdc[23107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:26.836004 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.818698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.1.8.126:40241)
0806 22:30:26.821316 (+ 2618us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.821341 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.821360 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:30:26.821407 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.821412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.821472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.821561 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:26.823296 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.824239 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.825882 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.826569 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.826714 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.829748 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:26.829757 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:26.829765 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:26.829804 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:26.833190 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.833902 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.833905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.833907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.834069 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.834454 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.834458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.834460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.834774 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:26.835029 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.835354 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.835592 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2515,"spinlock_wait_cycles":17792,"thread_start_us":2435,"threads_started":1}
I20260806 22:30:26.836464 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.816225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:59027 (local address 127.1.8.126:40241)
0806 22:30:26.816412 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.816417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.817535 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 22:30:26.820422 (+ 2887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.820428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.820514 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.820593 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:26.822507 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.823479 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.825250 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.825904 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.826017 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.829855 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:26.829859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:26.829862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:26.829905 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:26.832984 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.833779 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.833785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.833788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.833934 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:26.834444 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:26.834449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:26.834452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:26.834769 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:26.834914 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.835822 (+ 908us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.835969 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:30:26.837442 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.816688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:59027)
0806 22:30:26.817367 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:26.817412 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:26.820038 (+ 2626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:26.820975 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:26.820982 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:26.821463 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:26.822296 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.822324 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.823650 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.823654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.825109 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.825121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.826128 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.826135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.826415 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.827195 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:26.827217 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:26.829634 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:26.833143 (+ 3509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:26.833151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:26.833165 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:26.833632 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:26.834080 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:26.834084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:26.834087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:26.834312 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:26.834901 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:26.834909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:26.835300 (+ 391us) client_negotiation.cc:769] Sending connection context
0806 22:30:26.835597 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 22:30:26.836134 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":274,"threads_started":1}
I20260806 22:30:26.838861 24669 heartbeater.cc:347] Connected to a master server at 127.1.8.126:40241
I20260806 22:30:26.839123 24669 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:26.839788 24669 heartbeater.cc:511] Master 127.1.8.126:40241 requested a full tablet report, sending...
I20260806 22:30:26.841120 24002 ts_manager.cc:194] Registered new tserver with Master: 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041)
I20260806 22:30:26.841914 24002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:59027
I20260806 22:30:26.850692 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:30:26.857117 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.851794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.1.8.126:40241)
0806 22:30:26.851960 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.851964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.852399 (+ 435us) server_negotiation.cc:407] Connection header received
0806 22:30:26.852543 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.852547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.852672 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.852804 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:30:26.853184 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.853865 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.855274 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.855671 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.855758 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.856259 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:30:26.856690 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:30:26.856788 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.856893 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.856948 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:30:26.857944 24002 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:30:26.864943 24002 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:30:26.890859 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.876169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.1.8.66:46351)
0806 22:30:26.876787 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.876798 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.877512 (+ 714us) server_negotiation.cc:407] Connection header received
0806 22:30:26.880030 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.880038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.880300 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.880463 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:26.882380 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.883895 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.887772 (+ 3877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.889627 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.889770 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.889952 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.890320 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.890505 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":314,"threads_started":1}
I20260806 22:30:26.891055 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.875878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:46351 (local address 127.0.0.1:43670)
0806 22:30:26.877342 (+ 1464us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:26.877395 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:26.879735 (+ 2340us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x61200002cc28. stack: 00007f30f0c8766e 00007f30f0c87430 00007f30efd0e5ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f30f0c7d0fd 00007f30f0c7cd1e 00007f3103b863cd 00007f3103b8636c 00007f3103b52b3b 00007f30f768c536 00007f30f768ad9e 00007f30f7712dca 00007f30f7710f68 00007f30f774482f
0806 22:30:26.879793 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:26.880838 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:26.880845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:26.880883 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:26.882181 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.882196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.884246 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.884252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.887614 (+ 3362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.887630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.889783 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.889790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.890056 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.890116 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:30:26.890266 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 22:30:26.890432 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"mutex_wait_us":33,"spinlock_wait_cycles":3481856,"thread_start_us":1151,"threads_started":1}
I20260806 22:30:26.893817 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.876340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37041 (local address 127.0.0.1:40274)
0806 22:30:26.877342 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:26.877395 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:26.885547 (+ 8152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:26.886441 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:26.886445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:26.886470 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:26.886856 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.886867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.888780 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.888784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.891360 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.891376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.892897 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.892903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.893102 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.893145 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:30:26.893238 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:30:26.893365 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":1,"thread_start_us":1162,"threads_started":1}
I20260806 22:30:26.894572 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.877063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.1.8.67:37041)
0806 22:30:26.878893 (+ 1830us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.878900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.879024 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:30:26.885764 (+ 6740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.885772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.885976 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.886175 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:26.887062 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.888612 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.891497 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.892662 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.892796 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.893025 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.893311 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.894162 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1652,"thread_start_us":1474,"threads_started":1}
I20260806 22:30:26.895223 24377 tablet_service.cc:1467] Processing CreateTablet for tablet ab754667b81c4fc0951b60c6275a7265 (DEFAULT_TABLE table=test-table [id=9023ae77d5fb4fe8bb44f78584679091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:26.896648 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.877972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46848)
0806 22:30:26.887468 (+ 9496us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:26.887483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:26.887640 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:26.888270 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:26.888275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:26.888300 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:26.888638 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.888650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.890476 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.890481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.892884 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:26.892897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.895208 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.895215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:26.895392 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.895426 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:30:26.896221 (+ 795us) client_negotiation.cc:240] Negotiation successful
0806 22:30:26.896354 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9379,"thread_start_us":9216,"threads_started":1}
I20260806 22:30:26.896698 24377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab754667b81c4fc0951b60c6275a7265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:26.896968 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:26.878597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.1.8.65:44769)
0806 22:30:26.887170 (+ 8573us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:26.887180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:26.887588 (+ 408us) server_negotiation.cc:407] Connection header received
0806 22:30:26.887858 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:26.887866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:26.888048 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:26.888258 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:26.888865 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.890306 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.893066 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:26.894678 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:26.894813 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:26.895063 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:26.896286 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0806 22:30:26.896439 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8399,"thread_start_us":8270,"threads_started":1}
I20260806 22:30:26.897825 24574 tablet_service.cc:1467] Processing CreateTablet for tablet ab754667b81c4fc0951b60c6275a7265 (DEFAULT_TABLE table=test-table [id=9023ae77d5fb4fe8bb44f78584679091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:26.898931 24574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab754667b81c4fc0951b60c6275a7265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:26.903064 24177 tablet_service.cc:1467] Processing CreateTablet for tablet ab754667b81c4fc0951b60c6275a7265 (DEFAULT_TABLE table=test-table [id=9023ae77d5fb4fe8bb44f78584679091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:26.904438 24177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab754667b81c4fc0951b60c6275a7265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:26.910321 24689 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Bootstrap starting.
I20260806 22:30:26.912346 24690 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Bootstrap starting.
I20260806 22:30:26.914335 24689 tablet_bootstrap.cc:654] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:26.915627 24689 log.cc:824] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:26.915863 24690 tablet_bootstrap.cc:654] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:26.916963 24690 log.cc:824] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:26.918329 24284 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:26.918821 24689 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: No bootstrap required, opened a new log
I20260806 22:30:26.919109 24689 ts_tablet_manager.cc:1397] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 22:30:26.920255 24690 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: No bootstrap required, opened a new log
I20260806 22:30:26.920586 24690 ts_tablet_manager.cc:1397] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260806 22:30:26.920670 24695 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Bootstrap starting.
I20260806 22:30:26.923962 24695 tablet_bootstrap.cc:654] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:26.925217 24695 log.cc:824] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:26.929497 24695 tablet_bootstrap.cc:492] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: No bootstrap required, opened a new log
I20260806 22:30:26.929765 24695 ts_tablet_manager.cc:1397] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 22:30:26.929697 24689 raft_consensus.cc:348] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.930146 24689 raft_consensus.cc:374] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:26.930265 24689 raft_consensus.cc:729] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5665b1f494ea4d7199b4f20eedc66f56, State: Initialized, Role: FOLLOWER
I20260806 22:30:26.930114 24690 raft_consensus.cc:348] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.930538 24690 raft_consensus.cc:374] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:26.930653 24690 raft_consensus.cc:729] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e511ac4bd71f45828f040cd0a2036970, State: Initialized, Role: FOLLOWER
I20260806 22:30:26.930863 24689 consensus_queue.cc:260] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.931280 24690 consensus_queue.cc:260] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.932135 24669 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:26.932693 24689 ts_tablet_manager.cc:1428] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260806 22:30:26.932735 24480 heartbeater.cc:503] Master 127.1.8.126:40241 was elected leader, sending a full tablet report...
I20260806 22:30:26.933360 24690 ts_tablet_manager.cc:1428] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Time spent starting tablet: real 0.013s user 0.008s sys 0.003s
I20260806 22:30:26.940430 24695 raft_consensus.cc:348] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.940755 24695 raft_consensus.cc:374] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:26.940837 24695 raft_consensus.cc:729] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ac967012ef40f6bce314fd4fd41f6a, State: Initialized, Role: FOLLOWER
I20260806 22:30:26.941291 24695 consensus_queue.cc:260] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:26.942767 24695 ts_tablet_manager.cc:1428] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 22:30:27.013906 24698 raft_consensus.cc:482] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:30:27.014273 24698 raft_consensus.cc:504] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:27.015878 24698 leader_election.cc:288] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041), 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769)
I20260806 22:30:27.025321 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.016479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:48441 (local address 127.1.8.65:44769)
0806 22:30:27.016711 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.016719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.017374 (+ 655us) server_negotiation.cc:407] Connection header received
0806 22:30:27.017503 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.017509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.017601 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.017707 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:27.018589 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.019737 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.022860 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.024149 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.024286 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.024390 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.024935 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.025061 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:30:27.025311 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.016480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:35099 (local address 127.1.8.67:37041)
0806 22:30:27.016847 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.016854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.017290 (+ 436us) server_negotiation.cc:407] Connection header received
0806 22:30:27.017505 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.017510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.017591 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.017707 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:27.018586 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.019737 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.022860 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.024167 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.024285 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.024390 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.024936 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.025061 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260806 22:30:27.025760 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.016288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:37041 (local address 127.1.8.66:35099)
0806 22:30:27.017163 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.017184 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:27.017359 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:27.017749 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:27.017754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:27.017781 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:27.018415 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:27.018430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.019873 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.019879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:27.022684 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:27.022700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.024436 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.024440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:27.024745 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.024794 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:30:27.024892 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:30:27.025053 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":331,"thread_start_us":419,"threads_started":1}
I20260806 22:30:27.025840 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.016387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.1.8.66:48441)
0806 22:30:27.017266 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.017281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:27.017377 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:27.017749 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:27.017754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:27.017841 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:27.018422 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:27.018430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.019875 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.019879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:27.022684 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:27.022700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.024433 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.024440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:27.024748 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.024796 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:30:27.024892 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:30:27.025053 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":342,"spinlock_wait_cycles":20352,"thread_start_us":549,"threads_started":1}
I20260806 22:30:27.026996 24606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab754667b81c4fc0951b60c6275a7265" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665b1f494ea4d7199b4f20eedc66f56" is_pre_election: true
I20260806 22:30:27.027034 24210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab754667b81c4fc0951b60c6275a7265" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ac967012ef40f6bce314fd4fd41f6a" is_pre_election: true
I20260806 22:30:27.027607 24606 raft_consensus.cc:2454] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 0.
I20260806 22:30:27.027611 24210 raft_consensus.cc:2454] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 0.
I20260806 22:30:27.028432 24318 leader_election.cc:302] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03ac967012ef40f6bce314fd4fd41f6a, 5665b1f494ea4d7199b4f20eedc66f56, e511ac4bd71f45828f040cd0a2036970; no voters:
I20260806 22:30:27.028903 24698 raft_consensus.cc:2781] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:30:27.029073 24698 raft_consensus.cc:482] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:30:27.029196 24698 raft_consensus.cc:3037] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:27.031395 24698 raft_consensus.cc:504] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:27.031977 24698 leader_election.cc:288] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 election: Requested vote from peers 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041), 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769)
I20260806 22:30:27.032459 24606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab754667b81c4fc0951b60c6275a7265" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665b1f494ea4d7199b4f20eedc66f56"
I20260806 22:30:27.032478 24210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab754667b81c4fc0951b60c6275a7265" candidate_uuid: "e511ac4bd71f45828f040cd0a2036970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ac967012ef40f6bce314fd4fd41f6a"
I20260806 22:30:27.032647 24606 raft_consensus.cc:3037] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:27.032680 24210 raft_consensus.cc:3037] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:27.034822 24210 raft_consensus.cc:2454] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 1.
I20260806 22:30:27.034842 24606 raft_consensus.cc:2454] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e511ac4bd71f45828f040cd0a2036970 in term 1.
I20260806 22:30:27.035240 24318 leader_election.cc:302] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03ac967012ef40f6bce314fd4fd41f6a, e511ac4bd71f45828f040cd0a2036970; no voters:
I20260806 22:30:27.035513 24698 raft_consensus.cc:2781] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:27.035949 24698 raft_consensus.cc:686] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 1 LEADER]: Becoming Leader. State: Replica: e511ac4bd71f45828f040cd0a2036970, State: Running, Role: LEADER
I20260806 22:30:27.036439 24698 consensus_queue.cc:237] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } }
I20260806 22:30:27.040714 24001 catalog_manager.cc:5495] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 reported cstate change: term changed from 0 to 1, leader changed from <none> to e511ac4bd71f45828f040cd0a2036970 (127.1.8.66). New cstate: current_term: 1 leader_uuid: "e511ac4bd71f45828f040cd0a2036970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e511ac4bd71f45828f040cd0a2036970" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 46351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 } health_report { overall_health: UNKNOWN } } }
W20260806 22:30:27.065268 24670 tablet.cc:2369] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:27.076164 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.071266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.1.8.66:46351)
0806 22:30:27.071449 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.071454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.071472 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:30:27.071630 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.071634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.071707 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.071822 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:30:27.072182 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.072848 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.074129 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.074575 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.074666 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.075090 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:30:27.075551 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:30:27.075654 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.075739 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.075815 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:30:27.083938 24210 raft_consensus.cc:1261] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Refusing update from remote peer e511ac4bd71f45828f040cd0a2036970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:27.083940 24606 raft_consensus.cc:1261] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Refusing update from remote peer e511ac4bd71f45828f040cd0a2036970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:27.084932 24703 consensus_queue.cc:1035] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5665b1f494ea4d7199b4f20eedc66f56" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:27.085253 24698 consensus_queue.cc:1035] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03ac967012ef40f6bce314fd4fd41f6a" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:27.109790 24002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:27.110060 24002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:27.113569 24002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Sending DeleteTablet for 3 replicas of tablet ab754667b81c4fc0951b60c6275a7265
I20260806 22:30:27.114706 24377 tablet_service.cc:1514] Processing DeleteTablet for tablet ab754667b81c4fc0951b60c6275a7265 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:27 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:43670
I20260806 22:30:27.114727 24574 tablet_service.cc:1514] Processing DeleteTablet for tablet ab754667b81c4fc0951b60c6275a7265 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:27 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:40274
I20260806 22:30:27.115468 24177 tablet_service.cc:1514] Processing DeleteTablet for tablet ab754667b81c4fc0951b60c6275a7265 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:27 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:46848
I20260806 22:30:27.115592 24719 tablet_replica.cc:330] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: stopping tablet replica
I20260806 22:30:27.116124 24719 raft_consensus.cc:2229] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:27.116709 24719 raft_consensus.cc:2258] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:27.116897 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 24055
I20260806 22:30:27.119030 24719 ts_tablet_manager.cc:1905] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:27.121800 24721 tablet_replica.cc:330] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: stopping tablet replica
I20260806 22:30:27.122005 24720 tablet_replica.cc:330] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: stopping tablet replica
I20260806 22:30:27.122202 24721 raft_consensus.cc:2229] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:27.122359 24720 raft_consensus.cc:2229] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:30:27.122983 24720 raft_consensus.cc:2258] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:27.126861 24720 ts_tablet_manager.cc:1905] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:27.127924 24719 ts_tablet_manager.cc:1918] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:27.128131 24719 log.cc:1196] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/ab754667b81c4fc0951b60c6275a7265
I20260806 22:30:27.128697 24719 ts_tablet_manager.cc:1939] T ab754667b81c4fc0951b60c6275a7265 P 5665b1f494ea4d7199b4f20eedc66f56: Deleting consensus metadata
I20260806 22:30:27.130530 23981 catalog_manager.cc:4841] TS 5665b1f494ea4d7199b4f20eedc66f56 (127.1.8.67:37041): tablet ab754667b81c4fc0951b60c6275a7265 (table test-table [id=9023ae77d5fb4fe8bb44f78584679091]) successfully deleted
I20260806 22:30:27.132459 24721 raft_consensus.cc:2258] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:27.134757 24721 ts_tablet_manager.cc:1905] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:27.135417 24720 ts_tablet_manager.cc:1918] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:27.135581 24720 log.cc:1196] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/ab754667b81c4fc0951b60c6275a7265
I20260806 22:30:27.136221 24720 ts_tablet_manager.cc:1939] T ab754667b81c4fc0951b60c6275a7265 P e511ac4bd71f45828f040cd0a2036970: Deleting consensus metadata
I20260806 22:30:27.137909 23984 catalog_manager.cc:4841] TS e511ac4bd71f45828f040cd0a2036970 (127.1.8.66:46351): tablet ab754667b81c4fc0951b60c6275a7265 (table test-table [id=9023ae77d5fb4fe8bb44f78584679091]) successfully deleted
I20260806 22:30:27.138216 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.119978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.1.8.65:44769)
0806 22:30:27.120228 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.120236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.120258 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:30:27.120327 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.120334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.120457 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.120574 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:27.122891 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.124023 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.125392 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.126050 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.126174 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.129548 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:27.129562 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:27.129577 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:27.129615 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:27.133576 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.134904 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.134911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.134914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.135073 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.135565 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.135570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.135573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.136030 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:27.136476 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.137622 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.137846 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:30:27.138854 24273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:27.249516 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 24055
I20260806 22:30:27.274603 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 24289
W20260806 22:30:27.274557 23984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:44769 (error 108)
I20260806 22:30:27.276696 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.276198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46868)
0806 22:30:27.276380 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.276514 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 22:30:27.276983 23984 catalog_manager.cc:4584] TS 03ac967012ef40f6bce314fd4fd41f6a (127.1.8.65:44769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab754667b81c4fc0951b60c6275a7265: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
I20260806 22:30:27.291375 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.275873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.1.8.66:46351)
0806 22:30:27.276152 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.276158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.276182 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:30:27.276258 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.276263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.276358 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.276483 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:27.278311 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.279144 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.281281 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.281733 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.281830 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.285216 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:27.285229 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:27.285244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:27.285285 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:27.288578 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.289410 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.289416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.289419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.289581 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.289993 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.289996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.289998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.290342 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:27.290626 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.290909 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.291074 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:30:27.292057 24474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:27.330754 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.330146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46884)
0806 22:30:27.330390 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.330539 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:30:27.331864 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.331448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46900)
0806 22:30:27.331621 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.331709 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:30:27.392560 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.391976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46904)
0806 22:30:27.392264 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.392374 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 22:30:27.393646 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.393309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46908)
0806 22:30:27.393464 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.393539 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:30:27.420989 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 24289
I20260806 22:30:27.448894 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 24484
I20260806 22:30:27.462287 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.449727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.1.8.67:37041)
0806 22:30:27.449918 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:27.449923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:27.449943 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:30:27.450109 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:27.450113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:27.450189 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:27.450273 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:27.451618 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.452385 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.453669 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:27.454025 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:27.454113 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:27.457076 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:27.457090 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:27.457107 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:27.457137 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:27.459687 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.460394 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.460401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.460403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.460554 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:27.460880 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:27.460883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:27.460885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:27.461311 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:27.461583 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:27.461780 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:30:27.461936 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:30:27.462949 24664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:27.474887 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.474410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46914)
0806 22:30:27.474621 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.474703 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:30:27.475921 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.475535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46926)
0806 22:30:27.475729 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.475793 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 22:30:27.552242 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 24484
I20260806 22:30:27.575992 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 23956
I20260806 22:30:27.577314 24046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:27.693234 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:27.692474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44769 (local address 127.0.0.1:46942)
0806 22:30:27.692746 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:27.692875 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:30:27.693357 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 23956
I20260806 22:30:27.724192 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--webserver_interface=127.1.8.126
--webserver_port=45047
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:28.084800 24731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:28.085094 24731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:28.085246 24731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:28.092296 24731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:28.092430 24731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:28.092540 24731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:28.092639 24731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:28.092725 24731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:28.102586 24731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=45047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:28.103946 24731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:28.105347 24731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:28.117134 24737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:28.117167 24739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:28.117062 24742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:28.118165 24731 server_base.cc:1034] running on GCE node
I20260806 22:30:28.118875 24731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:28.120283 24731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:28.121490 24731 hybrid_clock.cc:648] HybridClock initialized: now 1786055428121425 us; error 62 us; skew 500 ppm
I20260806 22:30:28.121841 24731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:28 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055428, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:28.125022 24731 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:28.126629 24731 webserver.cc:460] Webserver started at http://127.1.8.126:45047/ using document root <none> and password file <none>
I20260806 22:30:28.127331 24731 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:28.127463 24731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:28.133122 24731 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:30:28.135972 24754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:28.137374 24731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:30:28.137555 24731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "605f0e3dee16466f8c6b677fa9a7b598"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "5fb469fd6dafb9976c6c98761755d39e"
server_key_iv: "1a7b1b2e56e245b4de11c9c2d226f107"
server_key_version: "encryptionkey@0"
I20260806 22:30:28.138554 24731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:28.156963 24731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:28.161023 24731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:28.161332 24731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:28.180770 24731 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:40241
I20260806 22:30:28.180809 24838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:40241 every 8 connection(s)
I20260806 22:30:28.182379 24731 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:28.185415 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 24731
I20260806 22:30:28.186524 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:44769
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=35871
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:28.191560 24839 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:30:28.196894 24839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap starting.
I20260806 22:30:28.219527 24839 log.cc:824] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:28.236377 24839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:30:28.236960 24839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598: Bootstrap complete.
I20260806 22:30:28.246533 24839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:28.247442 24839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Initialized, Role: FOLLOWER
I20260806 22:30:28.248080 24839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:28.248404 24839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:28.248530 24839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:28.248744 24839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:30:28.252754 24839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:28.253295 24839 leader_election.cc:302] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 605f0e3dee16466f8c6b677fa9a7b598; no voters:
I20260806 22:30:28.253997 24839 leader_election.cc:288] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:30:28.254129 24846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:30:28.255360 24846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [term 3 LEADER]: Becoming Leader. State: Replica: 605f0e3dee16466f8c6b677fa9a7b598, State: Running, Role: LEADER
I20260806 22:30:28.255978 24846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } }
I20260806 22:30:28.256338 24839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:28.258741 24849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:28.258903 24850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 605f0e3dee16466f8c6b677fa9a7b598. Latest consensus state: current_term: 3 leader_uuid: "605f0e3dee16466f8c6b677fa9a7b598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605f0e3dee16466f8c6b677fa9a7b598" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 40241 } } }
I20260806 22:30:28.259203 24849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:28.259227 24850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605f0e3dee16466f8c6b677fa9a7b598 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:28.267225 24853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:28.276163 24853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:28.277848 24853 catalog_manager.cc:1259] Loaded cluster ID: ddb40c2e590545088ea2a08e3765d8e2
I20260806 22:30:28.278012 24853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:30:28.280426 24853 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f61cca009ad google::LogMessage::Fail() at ??:0
@ 0x7f61cca0388a google::LogMessage::SendToLog() at ??:0
@ 0x7f61cca004de google::LogMessage::Flush() at ??:0
@ 0x7f61cca01a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f61e2433512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f61e2430976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f61e24b8610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f61e24b8439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61cf64c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f61cf64f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f61cf64f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61cf62aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f61d29046db start_thread at ??:0
@ 0x7f61cb89a61f clone at ??:0
W20260806 22:30:28.586138 24843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:28.586470 24843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:28.586647 24843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:28.594071 24843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:28.594233 24843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:28.594452 24843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:28.606451 24843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:44769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=35871
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:28.608332 24843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:28.610097 24843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:28.626363 24904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:28.626443 24903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:28.626446 24908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:28.627487 24843 server_base.cc:1034] running on GCE node
I20260806 22:30:28.628266 24843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:28.629290 24843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:28.630573 24843 hybrid_clock.cc:648] HybridClock initialized: now 1786055428630507 us; error 55 us; skew 500 ppm
I20260806 22:30:28.630985 24843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:28 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055428, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:28.634418 24843 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:28.635905 24843 webserver.cc:460] Webserver started at http://127.1.8.65:35871/ using document root <none> and password file <none>
I20260806 22:30:28.636695 24843 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:28.636835 24843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:28.642200 24843 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 22:30:28.645179 24920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:28.646525 24843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:30:28.646699 24843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "03ac967012ef40f6bce314fd4fd41f6a"
format_stamp: "Formatted at 2026-08-06 22:30:22 on dist-test-slave-7vjv"
server_key: "b718e940b6e46511b62573d6baab8277"
server_key_iv: "70991f4a6d070d8ac57336d664836078"
server_key_version: "encryptionkey@0"
I20260806 22:30:28.647672 24843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:28.679119 24843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:28.683856 24843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:28.684214 24843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:28.685385 24843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:28.687256 24932 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 22:30:28.692724 24932 ts_tablet_manager.cc:1852] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 22:30:28.693748 24932 ts_tablet_manager.cc:1905] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:28.701253 24932 ts_tablet_manager.cc:1918] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:28.701481 24932 log.cc:1196] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/ab754667b81c4fc0951b60c6275a7265
I20260806 22:30:28.701975 24932 ts_tablet_manager.cc:1939] T ab754667b81c4fc0951b60c6275a7265 P 03ac967012ef40f6bce314fd4fd41f6a: Deleting consensus metadata
I20260806 22:30:28.702379 24843 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 22:30:28.702508 24843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260806 22:30:28.702629 24843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:28.702732 24843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:28.738166 24843 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44769
I20260806 22:30:28.738261 25096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44769 every 8 connection(s)
I20260806 22:30:28.739614 24843 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:28.741693 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 24843
I20260806 22:30:28.742380 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:46351
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=45431
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:28.745191 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:28.743086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:46043)
0806 22:30:28.743990 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:28.744256 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":397,"threads_started":1}
W20260806 22:30:28.745992 25097 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:40241 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:40241: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
I20260806 22:30:28.746905 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:28.746559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:52715)
0806 22:30:28.746718 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:28.746797 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:30:28.747839 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:28.747485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:36339)
0806 22:30:28.747630 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:28.747712 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:30:28.748085 25097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:30:29.089608 25101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:29.089915 25101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:29.090083 25101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:29.097458 25101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:29.097599 25101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:29.097766 25101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:29.107268 25101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:46351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=45431
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:29.108609 25101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:29.109915 25101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:29.123271 25109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:29.123301 25110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:29.123502 25112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:29.124491 25101 server_base.cc:1034] running on GCE node
I20260806 22:30:29.125144 25101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:29.126024 25101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:29.127236 25101 hybrid_clock.cc:648] HybridClock initialized: now 1786055429127178 us; error 54 us; skew 500 ppm
I20260806 22:30:29.127660 25101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:29 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055429, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:29.131094 25101 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:29.132577 25101 webserver.cc:460] Webserver started at http://127.1.8.66:45431/ using document root <none> and password file <none>
I20260806 22:30:29.133383 25101 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:29.133531 25101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:29.138783 25101 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 22:30:29.141482 25120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.142747 25101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:30:29.142954 25101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "e511ac4bd71f45828f040cd0a2036970"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "1d3b3dfebe092d60623a8e1a9a010e84"
server_key_iv: "68b0b0a901fd24780c8e4b1b75d19941"
server_key_version: "encryptionkey@0"
I20260806 22:30:29.144028 25101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:29.171051 25101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:29.175277 25101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:29.175583 25101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:29.176656 25101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:29.178028 25101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:29.178162 25101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.178305 25101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:29.178399 25101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.212808 25101 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:46351
I20260806 22:30:29.212900 25305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:46351 every 8 connection(s)
I20260806 22:30:29.214315 25101 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:29.214686 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 25101
I20260806 22:30:29.215445 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:37041
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:40241
--webserver_port=43155
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:30:29.219782 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.217847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:52721)
0806 22:30:29.218644 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.218866 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":333,"threads_started":1}
W20260806 22:30:29.220414 25307 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:40241 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:40241: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
I20260806 22:30:29.221451 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.221023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:60979)
0806 22:30:29.221230 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.221317 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:30:29.222353 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.222049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:46453)
0806 22:30:29.222181 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.222251 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:30:29.222656 25307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:30:29.561280 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:29.561549 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:29.561726 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:29.568955 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:29.569090 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:29.569289 25311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:29.579118 25311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:37041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=43155
--tserver_master_addrs=127.1.8.126:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:29.580919 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:29.582939 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:29.598037 25326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:29.598055 25322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:29.598101 25323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:29.599439 25311 server_base.cc:1034] running on GCE node
I20260806 22:30:29.600018 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:29.600976 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:29.602212 25311 hybrid_clock.cc:648] HybridClock initialized: now 1786055429602164 us; error 34 us; skew 500 ppm
I20260806 22:30:29.602532 25311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:29 dist-test-slave-7vjv krb5kdc[23107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055429, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:29.605722 25311 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:29.607252 25311 webserver.cc:460] Webserver started at http://127.1.8.67:43155/ using document root <none> and password file <none>
I20260806 22:30:29.607985 25311 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:29.608157 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:29.613618 25311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 22:30:29.616464 25338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.617843 25311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:30:29.618026 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "5665b1f494ea4d7199b4f20eedc66f56"
format_stamp: "Formatted at 2026-08-06 22:30:23 on dist-test-slave-7vjv"
server_key: "fb7b72432594567b99b78b7f5cf4010f"
server_key_iv: "5bd3004798613857aa2bab70e92264e7"
server_key_version: "encryptionkey@0"
I20260806 22:30:29.619048 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:29.639191 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:29.643589 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:29.643903 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:29.645133 25311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:29.646556 25311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:29.646692 25311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.646832 25311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:29.646940 25311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:29.682149 25311 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:37041
I20260806 22:30:29.682236 25501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:37041 every 8 connection(s)
I20260806 22:30:29.683754 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:29.688690 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.686684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:50055)
0806 22:30:29.687528 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.687722 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":415,"threads_started":1}
W20260806 22:30:29.689365 25504 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:40241 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:40241: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
I20260806 22:30:29.690184 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.689872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:36915)
0806 22:30:29.690017 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.690081 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:30:29.690721 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 25311
I20260806 22:30:29.691090 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.690699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:59189)
0806 22:30:29.690866 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.690956 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:30:29.691391 25504 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:30:29.749325 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:29.748548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:48757)
0806 22:30:29.748994 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:29.749087 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":202,"threads_started":1}
I20260806 22:30:30.224253 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:30.223269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:53785)
0806 22:30:30.223827 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:30.223933 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":228,"threads_started":1}
I20260806 22:30:30.692783 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:30.691961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:45281)
0806 22:30:30.692465 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:30.692550 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":272,"threads_started":1}
I20260806 22:30:30.750933 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:30.750201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:49333)
0806 22:30:30.750702 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:30.750779 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":290,"threads_started":1}
I20260806 22:30:31.225780 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:31.225072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:59171)
0806 22:30:31.225528 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:31.225608 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":222,"threads_started":1}
I20260806 22:30:31.694298 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:31.693563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:45357)
0806 22:30:31.694042 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:31.694129 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
I20260806 22:30:31.752635 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:31.751841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:39003)
0806 22:30:31.752360 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:31.752443 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":265,"threads_started":1}
I20260806 22:30:32.227516 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:32.226749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:59571)
0806 22:30:32.227266 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:32.227345 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":320,"threads_started":1}
I20260806 22:30:32.695958 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:32.695247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:60431)
0806 22:30:32.695706 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:32.695786 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":237,"threads_started":1}
I20260806 22:30:32.754218 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:32.753480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:42437)
0806 22:30:32.753968 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:32.754051 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":244,"threads_started":1}
I20260806 22:30:33.229291 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:33.228531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:41333)
0806 22:30:33.229016 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:33.229101 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260806 22:30:33.697824 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:33.696874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:47351)
0806 22:30:33.697533 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:33.697617 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":346,"threads_started":1}
I20260806 22:30:33.755751 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:33.755076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:53483)
0806 22:30:33.755500 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:33.755582 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":218,"threads_started":1}
I20260806 22:30:34.230955 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:34.230221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:39197)
0806 22:30:34.230710 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:34.230788 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":277,"threads_started":1}
I20260806 22:30:34.699519 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:34.698766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:58723)
0806 22:30:34.699244 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:34.699328 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":251,"threads_started":1}
I20260806 22:30:34.757473 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:34.756709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:42193)
0806 22:30:34.757200 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:34.757290 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260806 22:30:35.232734 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:35.231867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:43749)
0806 22:30:35.232446 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:35.232565 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":342,"threads_started":1}
I20260806 22:30:35.701133 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:35.700447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:37485)
0806 22:30:35.700882 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:35.700971 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":224,"threads_started":1}
I20260806 22:30:35.759481 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:35.758481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:35155)
0806 22:30:35.759103 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:35.759189 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":288,"threads_started":1}
I20260806 22:30:36.234539 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:36.233666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:40051)
0806 22:30:36.234225 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:36.234310 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":276,"threads_started":1}
I20260806 22:30:36.702831 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:36.702005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:37317)
0806 22:30:36.702527 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:36.702607 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":239,"threads_started":1}
I20260806 22:30:36.761078 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:36.760393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:57455)
0806 22:30:36.760840 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:36.760915 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":226,"threads_started":1}
I20260806 22:30:37.236260 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:37.235510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:43171)
0806 22:30:37.235952 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:37.236033 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260806 22:30:37.704666 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:37.703918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:55399)
0806 22:30:37.704405 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:37.704490 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":278,"threads_started":1}
I20260806 22:30:37.762877 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:37.762116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:33699)
0806 22:30:37.762611 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:37.762711 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":232,"threads_started":1}
I20260806 22:30:38.238078 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:38.237162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:52805)
0806 22:30:38.237820 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:38.237905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":391,"threads_started":1}
I20260806 22:30:38.706357 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:38.705609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:52007)
0806 22:30:38.706088 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:38.706173 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":258,"threads_started":1}
I20260806 22:30:38.764705 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:38.763981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:51223)
0806 22:30:38.764464 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:38.764540 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":290,"threads_started":1}
I20260806 22:30:39.239771 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:39.239013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:35895)
0806 22:30:39.239487 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:39.239570 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260806 22:30:39.708130 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:39.707400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:42271)
0806 22:30:39.707839 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:39.707916 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":243,"threads_started":1}
I20260806 22:30:39.766474 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:39.765665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:41351)
0806 22:30:39.766207 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:39.766285 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":321,"threads_started":1}
I20260806 22:30:40.241559 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:40.240694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:57717)
0806 22:30:40.241180 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:40.241308 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":235,"threads_started":1}
I20260806 22:30:40.709884 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:40.709111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:47351)
0806 22:30:40.709608 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:40.709697 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":259,"threads_started":1}
I20260806 22:30:40.768216 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:40.767458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:34359)
0806 22:30:40.767887 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:40.767976 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":229,"threads_started":1}
I20260806 22:30:41.243317 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:41.242577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:41739)
0806 22:30:41.243069 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:41.243157 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260806 22:30:41.711726 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:41.710931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:41221)
0806 22:30:41.711478 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:41.711563 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
I20260806 22:30:41.769871 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:41.769136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:46941)
0806 22:30:41.769591 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:41.769669 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260806 22:30:42.245170 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:42.244458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:38943)
0806 22:30:42.244903 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:42.244986 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":220,"threads_started":1}
I20260806 22:30:42.713502 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:42.712624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:53963)
0806 22:30:42.713127 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:42.713211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":271,"threads_started":1}
I20260806 22:30:42.771642 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:42.770821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:44539)
0806 22:30:42.771354 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:42.771446 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260806 22:30:43.246850 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:43.246132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:50335)
0806 22:30:43.246578 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:43.246661 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260806 22:30:43.715219 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:43.714499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:36439)
0806 22:30:43.714921 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:43.715031 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":225,"threads_started":1}
I20260806 22:30:43.773524 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:43.772769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:36097)
0806 22:30:43.773236 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:43.773314 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
I20260806 22:30:44.248925 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.248188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.66:50373)
0806 22:30:44.248656 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:44.248740 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":221,"threads_started":1}
I20260806 22:30:44.692878 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 24843
I20260806 22:30:44.715205 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.694185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.1.8.65:44769)
0806 22:30:44.694775 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:44.694790 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:44.694852 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:30:44.697113 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:44.697123 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:44.697584 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:44.697939 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:44.699434 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.700813 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.702119 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.702768 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.702974 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:44.708793 (+ 5819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:44.708801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:44.708817 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:44.708862 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:44.711664 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.712476 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.712484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.712490 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.712659 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.713074 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.713079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.713082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.713620 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:44.713979 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:44.714491 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 22:30:44.714915 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":290,"threads_started":1}
I20260806 22:30:44.716817 25091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:44.717612 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.716523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.67:58395)
0806 22:30:44.717039 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:44.717182 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":242,"threads_started":1}
I20260806 22:30:44.801350 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.800186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:40241 (local address 127.1.8.65:37715)
0806 22:30:44.800767 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:44.801007 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":274,"threads_started":1}
I20260806 22:30:44.804278 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 24843
I20260806 22:30:44.833679 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 25101
I20260806 22:30:44.851805 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.834682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.1.8.66:46351)
0806 22:30:44.835243 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:44.835257 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:44.835298 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:30:44.836989 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:44.837000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:44.837451 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:44.837742 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:44.839044 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.840530 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.841891 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.842570 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.842731 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:44.845402 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:44.845411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:44.845427 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:44.845467 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:44.848693 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.849375 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.849382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.849388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.849570 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.849931 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.849934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.849936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.850487 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:44.850740 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:44.851070 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:30:44.851503 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":290,"threads_started":1}
I20260806 22:30:44.853279 25299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:44.938099 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 25101
I20260806 22:30:44.967739 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 25311
I20260806 22:30:44.985042 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:44.968729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.1.8.67:37041)
0806 22:30:44.969354 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:44.969368 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:44.969413 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:30:44.971090 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:44.971100 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:44.971518 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:44.971828 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:44.973413 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.974619 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.975803 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:44.976499 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:44.976655 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:44.979162 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:44.979169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:44.979183 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:44.979219 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:44.982157 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.982792 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.982798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.982804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.982959 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:44.983272 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:44.983275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:44.983277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:44.983773 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:44.983999 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:44.984409 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 22:30:44.984745 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":322,"threads_started":1}
I20260806 22:30:44.986485 25497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:45.070669 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 25311
2026-08-06T22:30:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25026 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:30:45 dist-test-slave-7vjv krb5kdc[25591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:30:45 dist-test-slave-7vjv krb5kdc[25591](info): set up 2 sockets
Aug 06 22:30:45 dist-test-slave-7vjv krb5kdc[25591](info): commencing operation
krb5kdc: starting...
W20260806 22:30:46.434984 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:30:46 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:30:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:30:46.735769 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:47.081763 25608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:47.082057 25608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:47.082158 25608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:47.089339 25608 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:30:47.089448 25608 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:30:47.089488 25608 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:30:47.089541 25608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:30:47.089596 25608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:47.089664 25608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:47.089710 25608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:30:47.089761 25608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:30:47.099534 25608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:47.100862 25608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:47.102202 25608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:47.114199 25616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:47.114338 25619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:47.114279 25615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:47.115240 25608 server_base.cc:1034] running on GCE node
I20260806 22:30:47.116047 25608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:47.117660 25608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:47.118928 25608 hybrid_clock.cc:648] HybridClock initialized: now 1786055447118855 us; error 52 us; skew 500 ppm
I20260806 22:30:47.119313 25608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:47 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055447, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:47.122504 25608 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:30:47.124150 25608 webserver.cc:460] Webserver started at http://127.1.8.126:42625/ using document root <none> and password file <none>
I20260806 22:30:47.124878 25608 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:47.125003 25608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:47.125340 25608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:47.128419 25608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "69ebfbc4ca564dad9333f1b0b3853923"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "cce53dc66d784225f7f542bfc5b90d05"
server_key_iv: "ee023e422dcc00d151dd344326f7bf0c"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.129280 25608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "69ebfbc4ca564dad9333f1b0b3853923"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "cce53dc66d784225f7f542bfc5b90d05"
server_key_iv: "ee023e422dcc00d151dd344326f7bf0c"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.134488 25608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:30:47.137914 25629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:47.139122 25608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:30:47.139298 25608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "69ebfbc4ca564dad9333f1b0b3853923"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "cce53dc66d784225f7f542bfc5b90d05"
server_key_iv: "ee023e422dcc00d151dd344326f7bf0c"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.139516 25608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:47.158188 25608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:47.162122 25608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:47.162403 25608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:47.181739 25608 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42857
I20260806 22:30:47.181756 25720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42857 every 8 connection(s)
I20260806 22:30:47.183194 25608 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:30:47.184577 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 25608
I20260806 22:30:47.184820 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:30:47.185688 1057 external_mini_cluster.cc:1428] Setting key e6cf17ec4752680fdddf6895ef93272f
I20260806 22:30:47.188145 25721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:30:47 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:47.201601 25721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap starting.
I20260806 22:30:47.206079 25721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:47.207336 25721 log.cc:824] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:47.210388 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:47.187775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.1.8.126:42857)
0806 22:30:47.188594 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:47.188606 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:47.188646 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:30:47.190274 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:47.190294 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:47.190667 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:47.190985 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:47.192990 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.194336 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.196195 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.196834 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.196953 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:47.201754 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:47.201765 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:47.201783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:47.201825 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:47.205855 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:47.206600 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:47.206608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:47.206616 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:47.206820 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:47.207195 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:47.207201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:47.207204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:47.207892 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:47.208246 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:47.208679 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 22:30:47.209102 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":415,"threads_started":1}
I20260806 22:30:47.210752 25721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: No bootstrap required, opened a new log
I20260806 22:30:47.218447 25721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:30:47.218796 25721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:47.218937 25721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Initialized, Role: FOLLOWER
I20260806 22:30:47.219544 25721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:30:47.219821 25721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:30:47.219964 25721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:30:47.220152 25721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:47.222210 25721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:30:47.222697 25721 leader_election.cc:302] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ebfbc4ca564dad9333f1b0b3853923; no voters:
I20260806 22:30:47.223378 25721 leader_election.cc:288] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:30:47.223517 25728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:47.224257 25728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 LEADER]: Becoming Leader. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Running, Role: LEADER
I20260806 22:30:47.224840 25728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:30:47.225584 25721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:30:47.227660 25734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ebfbc4ca564dad9333f1b0b3853923. Latest consensus state: current_term: 1 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:30:47.228170 25732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:30:47.228478 25732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:47.228166 25734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:30:47.239174 25746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:30:47.244488 25746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:30:47.253588 25746 catalog_manager.cc:1347] Generated new cluster ID: ec4b1d788b0246d1b39f3a208bb372ac
I20260806 22:30:47.253729 25746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:30:47.290300 25746 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:30:47.291698 25746 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:30:47.299228 25746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Generated new TSK 0
I20260806 22:30:47.300362 25746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:30:47.387024 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:47.734402 25762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:47.734660 25762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:47.734809 25762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:47.742089 25762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:47.742209 25762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:47.742396 25762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:30:47.752138 25762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:47.753381 25762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:47.754680 25762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:47.767778 25768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:47.767824 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:47.767970 25772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:47.768550 25762 server_base.cc:1034] running on GCE node
I20260806 22:30:47.769160 25762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:47.770043 25762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:47.771265 25762 hybrid_clock.cc:648] HybridClock initialized: now 1786055447771218 us; error 58 us; skew 500 ppm
I20260806 22:30:47.771569 25762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:47 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055447, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:47.774863 25762 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:30:47.776247 25762 webserver.cc:460] Webserver started at http://127.1.8.65:42847/ using document root <none> and password file <none>
I20260806 22:30:47.776937 25762 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:47.777086 25762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:47.777402 25762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:47.780222 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "6b723a166b064107a91f1763689a105c"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "2c863e457fc14d552aa49684053a2b2d"
server_key_iv: "ddf741eca077e2018edd09555eb96b40"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.780977 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "6b723a166b064107a91f1763689a105c"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "2c863e457fc14d552aa49684053a2b2d"
server_key_iv: "ddf741eca077e2018edd09555eb96b40"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.785856 25762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:30:47.789359 25782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:47.790591 25762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:30:47.790767 25762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "6b723a166b064107a91f1763689a105c"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "2c863e457fc14d552aa49684053a2b2d"
server_key_iv: "ddf741eca077e2018edd09555eb96b40"
server_key_version: "encryptionkey@0"
I20260806 22:30:47.790977 25762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:47.808998 25762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:47.812780 25762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:47.813054 25762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:47.814005 25762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:47.815325 25762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:47.815428 25762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:47.815579 25762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:47.815650 25762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:47.849866 25762 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:37403
I20260806 22:30:47.849910 25949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:37403 every 8 connection(s)
I20260806 22:30:47.851318 25762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:30:47.858711 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 25762
I20260806 22:30:47.858958 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:30:47.859834 1057 external_mini_cluster.cc:1428] Setting key 06ac146f55eb677f008ebcae2f100107
Aug 06 22:30:47 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055447, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:47.873824 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:47.853909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:55813 (local address 127.1.8.126:42857)
0806 22:30:47.854426 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:47.854432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:47.855438 (+ 1006us) server_negotiation.cc:407] Connection header received
0806 22:30:47.857374 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:47.857380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:47.857483 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:47.857612 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:47.859727 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.860643 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.862700 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.863097 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.863189 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:47.867785 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:47.867789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:47.867792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:47.867825 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:47.870465 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:47.871409 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:47.871413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:47.871415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:47.871530 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:47.872023 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:47.872026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:47.872028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:47.872336 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:47.872451 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:47.873367 (+ 916us) server_negotiation.cc:299] Negotiation successful
0806 22:30:47.873537 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":256,"threads_started":1}
I20260806 22:30:47.875458 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:47.854484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:55813)
0806 22:30:47.855264 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:47.855314 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:47.857085 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:47.857855 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:47.857864 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:47.858605 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:47.859476 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:47.859504 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.860836 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.860841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:47.862543 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:47.862560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:47.863332 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:47.863341 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:47.863680 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:47.864494 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:47.864536 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:47.867555 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:47.870608 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:47.870619 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:47.870640 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:47.871293 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:47.871649 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:47.871656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:47.871661 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:47.871915 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:47.872462 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:47.872472 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:47.872955 (+ 483us) client_negotiation.cc:769] Sending connection context
0806 22:30:47.873292 (+ 337us) client_negotiation.cc:240] Negotiation successful
0806 22:30:47.873893 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":282,"threads_started":1}
I20260806 22:30:47.877197 25950 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:30:47.877560 25950 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:47.878362 25950 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:30:47.880704 25655 ts_manager.cc:194] Registered new tserver with Master: 6b723a166b064107a91f1763689a105c (127.1.8.65:37403)
I20260806 22:30:47.882207 25655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:55813
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:30:47.929348 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:48.289717 25957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:48.290087 25957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:48.290256 25957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:48.298231 25957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:48.298380 25957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:48.298573 25957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:30:48.308419 25957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:48.309748 25957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:48.311203 25957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:48.324751 25972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:48.324872 25967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:48.324784 25968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:48.325631 25957 server_base.cc:1034] running on GCE node
I20260806 22:30:48.326293 25957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:48.327258 25957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:48.328480 25957 hybrid_clock.cc:648] HybridClock initialized: now 1786055448328419 us; error 56 us; skew 500 ppm
I20260806 22:30:48.328788 25957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:48 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:48.332239 25957 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:30:48.333765 25957 webserver.cc:460] Webserver started at http://127.1.8.66:40085/ using document root <none> and password file <none>
I20260806 22:30:48.334450 25957 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:48.334607 25957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:48.335014 25957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:48.339426 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "0f570ec1b3854ff585185f7e49cc96f4"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "b0d7603e97aa966654fa4ef2a6fcb3c3"
server_key_iv: "ab00765434a408a982f4d7c1e5a37a73"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.340564 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "0f570ec1b3854ff585185f7e49cc96f4"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "b0d7603e97aa966654fa4ef2a6fcb3c3"
server_key_iv: "ab00765434a408a982f4d7c1e5a37a73"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.346362 25957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:30:48.349798 25985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.351115 25957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:30:48.351250 25957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "0f570ec1b3854ff585185f7e49cc96f4"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "b0d7603e97aa966654fa4ef2a6fcb3c3"
server_key_iv: "ab00765434a408a982f4d7c1e5a37a73"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.351419 25957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:48.370885 25957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:48.375564 25957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:48.375821 25957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:48.376883 25957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:48.378233 25957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:48.378317 25957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.378500 25957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:48.378553 25957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.412837 25957 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40185
I20260806 22:30:48.412881 26148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40185 every 8 connection(s)
I20260806 22:30:48.414165 25957 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:30:48.422358 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 25957
I20260806 22:30:48.422506 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:30:48.422984 1057 external_mini_cluster.cc:1428] Setting key 9afd4a14bd80bc4c7ed064d88cd699e9
Aug 06 22:30:48 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:48.437944 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:48.416768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:46795 (local address 127.1.8.126:42857)
0806 22:30:48.417250 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:48.417256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:48.418256 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 22:30:48.420001 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:48.420005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:48.420310 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:48.420433 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:48.421872 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.422914 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.425358 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.425828 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.425925 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:48.430394 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:48.430404 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:48.430411 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:48.430461 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:48.434304 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:48.435119 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:48.435127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:48.435134 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:48.435316 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:48.435744 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:48.435753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:48.435759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:48.436207 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:48.436395 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:48.437006 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 22:30:48.437328 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":225,"threads_started":1}
I20260806 22:30:48.438591 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:48.417367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:46795)
0806 22:30:48.418090 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:48.418142 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:48.419763 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:48.420646 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:48.420653 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:48.421160 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:48.421691 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:48.421708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.423097 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.423105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:48.425007 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:48.425026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.426085 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.426094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:48.426485 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:48.427254 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:48.427301 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:48.430139 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:48.434530 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:48.434537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:48.434549 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:48.434988 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:48.435446 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:48.435450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:48.435452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:48.435627 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:48.436338 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:48.436344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:48.436645 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 22:30:48.436876 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 22:30:48.437313 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":282,"threads_started":1}
I20260806 22:30:48.440042 26150 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:30:48.440398 26150 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:48.441140 26150 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:30:48.442775 25655 ts_manager.cc:194] Registered new tserver with Master: 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185)
I20260806 22:30:48.444235 25655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:46795
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:30:48.495160 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:30:48.836723 26160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:30:48.836989 26160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:30:48.837163 26160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:30:48.844560 26160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:30:48.844660 26160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:30:48.844765 26160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:30:48.854203 26160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:30:48.855338 26160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:30:48.856664 26160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:30:48.869868 26167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:48.869787 26169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:30:48.869841 26172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:30:48.870680 26160 server_base.cc:1034] running on GCE node
I20260806 22:30:48.871320 26160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:30:48.872247 26160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:30:48.873464 26160 hybrid_clock.cc:648] HybridClock initialized: now 1786055448873427 us; error 48 us; skew 500 ppm
I20260806 22:30:48.873771 26160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:30:48 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:30:48.876978 26160 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:30:48.878346 26160 webserver.cc:460] Webserver started at http://127.1.8.67:37869/ using document root <none> and password file <none>
I20260806 22:30:48.879038 26160 fs_manager.cc:359] Metadata directory not provided
I20260806 22:30:48.879189 26160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:30:48.879518 26160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:30:48.882381 26160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "cf7b854f41d04c8c98b0af720ef28278"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "2657278a9763f894f1a7591ed92cac2d"
server_key_iv: "564b568d2aebf48de03b883d861cc8e0"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.883133 26160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "cf7b854f41d04c8c98b0af720ef28278"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "2657278a9763f894f1a7591ed92cac2d"
server_key_iv: "564b568d2aebf48de03b883d861cc8e0"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.885195 25950 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:30:48.888783 26160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 22:30:48.892072 26182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.893371 26160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:30:48.893500 26160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "cf7b854f41d04c8c98b0af720ef28278"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "2657278a9763f894f1a7591ed92cac2d"
server_key_iv: "564b568d2aebf48de03b883d861cc8e0"
server_key_version: "encryptionkey@0"
I20260806 22:30:48.893677 26160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:30:48.908835 26160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:30:48.912731 26160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:30:48.912990 26160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:30:48.913890 26160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:30:48.915241 26160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:30:48.915313 26160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.915402 26160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:30:48.915447 26160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:30:48.948266 26160 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:42329
I20260806 22:30:48.948303 26346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:42329 every 8 connection(s)
I20260806 22:30:48.949538 26160 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:30:48.956048 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 26160
I20260806 22:30:48.956329 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:30:48.956784 1057 external_mini_cluster.cc:1428] Setting key 0c7d0da0bd49d2bedb8d7334f3068607
Aug 06 22:30:48 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:30:48.969123 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:48.952227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:45621 (local address 127.1.8.126:42857)
0806 22:30:48.952753 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:48.952757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:48.953584 (+ 827us) server_negotiation.cc:407] Connection header received
0806 22:30:48.955448 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:48.955453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:48.955557 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:48.955678 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:48.957395 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.958218 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.959745 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.960214 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.960347 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:48.963553 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:48.963561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:48.963570 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:48.963601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:48.966023 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:48.966738 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:48.966750 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:48.966753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:48.966871 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:48.967296 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:48.967301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:48.967305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:48.967566 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:48.967726 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:48.968472 (+ 746us) server_negotiation.cc:299] Negotiation successful
0806 22:30:48.968706 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":261,"threads_started":1}
I20260806 22:30:48.970038 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:48.952709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:45621)
0806 22:30:48.953419 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:48.953455 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:48.955195 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:48.955936 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:48.955944 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:48.956644 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:30:48.957227 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:48.957244 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.958384 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.958388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:48.959578 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:48.959589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.960432 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.960439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:48.960672 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:48.961283 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:30:48.961304 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:30:48.963362 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:30:48.966169 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:48.966177 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:48.966192 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:48.966611 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:48.966984 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:30:48.966989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:30:48.966992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:30:48.967181 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:30:48.967693 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:30:48.967701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:30:48.968023 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 22:30:48.968377 (+ 354us) client_negotiation.cc:240] Negotiation successful
0806 22:30:48.968886 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":285,"threads_started":1}
I20260806 22:30:48.971401 26347 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:30:48.971707 26347 heartbeater.cc:464] Registering TS with master...
I20260806 22:30:48.972517 26347 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:30:48.973875 25655 ts_manager.cc:194] Registered new tserver with Master: cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:30:48.974699 25655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:45621
I20260806 22:30:48.984522 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:30:49.002380 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:48.989305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.1.8.126:42857)
0806 22:30:48.989472 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:48.989477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:48.989877 (+ 400us) server_negotiation.cc:407] Connection header received
0806 22:30:48.990029 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:48.990032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:48.990101 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:48.990184 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:48.991593 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.992381 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.993727 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:48.994140 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:48.994303 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:48.997369 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:48.997374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:48.997377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:48.997409 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:49.000029 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.000741 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.000744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.000747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.000864 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.001182 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.001185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.001187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.001497 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:49.001690 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.001939 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.002133 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:30:49.007332 25655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:30:49.014472 25655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:30:49.039865 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.027006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:45654)
0806 22:30:49.028209 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:49.028257 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:49.028426 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:49.029973 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:49.029998 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:49.030030 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:49.030515 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.030531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.033284 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.033292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.036559 (+ 3267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.036575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.038621 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.038627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.038861 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.038940 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 22:30:49.039086 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:30:49.039312 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":68,"thread_start_us":2519,"threads_started":1}
I20260806 22:30:49.039906 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.027358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.1.8.65:37403)
0806 22:30:49.029423 (+ 2065us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.029429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.029452 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:30:49.029526 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.029531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.029711 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.030756 (+ 1045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:49.031091 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.033096 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.036771 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.038416 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.038569 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.038819 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.039345 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.039522 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1928,"thread_start_us":1775,"threads_started":1}
I20260806 22:30:49.040704 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.027192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40185 (local address 127.0.0.1:53610)
0806 22:30:49.029823 (+ 2631us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:49.029840 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:49.030065 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:49.031152 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:49.031159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:49.031192 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:49.031686 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.031701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.034101 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.034108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.037602 (+ 3494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.037613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.039821 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.039827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.040021 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.040057 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:30:49.040180 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:30:49.040309 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2485,"mutex_wait_us":750,"thread_start_us":731,"threads_started":1}
I20260806 22:30:49.040745 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.027777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.1.8.66:40185)
0806 22:30:49.028964 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.028973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.029935 (+ 962us) server_negotiation.cc:407] Connection header received
0806 22:30:49.030316 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.030327 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.030609 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.030800 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:49.031926 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.033865 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.037772 (+ 3907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.039326 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.039449 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.039638 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.040251 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.040421 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":934,"threads_started":1}
I20260806 22:30:49.044436 26056 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 (DEFAULT_TABLE table=test-table [id=45e55dc6a0f14943bcf6f962618fdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:49.045177 25851 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 (DEFAULT_TABLE table=test-table [id=45e55dc6a0f14943bcf6f962618fdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:49.045876 26056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8888c0f7f0412f8e32926761e49aa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:49.046523 25851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8888c0f7f0412f8e32926761e49aa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:49.053221 26377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.029911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42329 (local address 127.0.0.1:56032)
0806 22:30:49.036897 (+ 6986us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:49.036938 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:49.037044 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:49.038961 (+ 1917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:49.038966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:49.038991 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:49.039405 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.039417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.041334 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.041339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.042865 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.042878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.052348 (+ 9470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.052355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.052544 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.052589 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:30:49.052699 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:30:49.052869 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6873,"thread_start_us":6715,"threads_started":1}
I20260806 22:30:49.053292 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.030300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.1.8.67:42329)
0806 22:30:49.038297 (+ 7997us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.038305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.038340 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:30:49.038434 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.038442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.038674 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.038969 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:49.039631 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.041151 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.050560 (+ 9409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.052120 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.052258 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.052501 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.052763 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.052940 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7820,"thread_start_us":7661,"threads_started":1}
I20260806 22:30:49.057417 26253 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 (DEFAULT_TABLE table=test-table [id=45e55dc6a0f14943bcf6f962618fdac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:30:49.058597 26379 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Bootstrap starting.
I20260806 22:30:49.059865 26253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8888c0f7f0412f8e32926761e49aa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:30:49.060611 26380 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Bootstrap starting.
I20260806 22:30:49.062563 26379 tablet_bootstrap.cc:654] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:49.064251 26380 tablet_bootstrap.cc:654] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:49.064785 26379 log.cc:824] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:49.065408 26380 log.cc:824] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:49.068009 26379 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: No bootstrap required, opened a new log
I20260806 22:30:49.068321 26379 ts_tablet_manager.cc:1397] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 22:30:49.068539 26380 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: No bootstrap required, opened a new log
I20260806 22:30:49.068806 26380 ts_tablet_manager.cc:1397] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 22:30:49.072384 26386 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Bootstrap starting.
I20260806 22:30:49.075907 26386 tablet_bootstrap.cc:654] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Neither blocks nor log segments found. Creating new log.
I20260806 22:30:49.076906 26379 raft_consensus.cc:348] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.077157 26386 log.cc:824] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Log is configured to *not* fsync() on all Append() calls
I20260806 22:30:49.077275 26379 raft_consensus.cc:374] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:49.077409 26379 raft_consensus.cc:729] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f570ec1b3854ff585185f7e49cc96f4, State: Initialized, Role: FOLLOWER
I20260806 22:30:49.078172 26379 consensus_queue.cc:260] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.079425 26150 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:30:49.080222 26379 ts_tablet_manager.cc:1428] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260806 22:30:49.080441 26386 tablet_bootstrap.cc:492] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: No bootstrap required, opened a new log
I20260806 22:30:49.080371 26380 raft_consensus.cc:348] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.080667 26386 ts_tablet_manager.cc:1397] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 22:30:49.080786 26380 raft_consensus.cc:374] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:49.080924 26380 raft_consensus.cc:729] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b723a166b064107a91f1763689a105c, State: Initialized, Role: FOLLOWER
I20260806 22:30:49.081478 26380 consensus_queue.cc:260] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.083196 26380 ts_tablet_manager.cc:1428] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 22:30:49.090464 26386 raft_consensus.cc:348] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.090842 26386 raft_consensus.cc:374] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:30:49.090945 26386 raft_consensus.cc:729] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7b854f41d04c8c98b0af720ef28278, State: Initialized, Role: FOLLOWER
I20260806 22:30:49.091475 26386 consensus_queue.cc:260] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.092555 26347 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:30:49.093187 26386 ts_tablet_manager.cc:1428] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
W20260806 22:30:49.103825 25952 tablet.cc:2369] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:30:49.166200 26151 tablet.cc:2369] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:30:49.201218 26348 tablet.cc:2369] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:30:49.213114 26389 raft_consensus.cc:482] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:30:49.213447 26389 raft_consensus.cc:504] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.215224 26389 leader_election.cc:288] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:30:49.224241 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.215771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:54603 (local address 127.1.8.67:42329)
0806 22:30:49.215978 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.215986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.216781 (+ 795us) server_negotiation.cc:407] Connection header received
0806 22:30:49.216999 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.217004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.217096 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.217190 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:49.218002 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.219140 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.222073 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.223323 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.223456 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.223559 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.223844 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.223970 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:30:49.224406 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.215610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42329 (local address 127.1.8.65:54603)
0806 22:30:49.216649 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:49.216670 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:49.216846 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:49.217230 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:49.217235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:49.217264 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:49.217840 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.217854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.219276 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.219282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.221905 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.221920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.223536 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.223541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.223654 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.223682 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 22:30:49.223804 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 22:30:49.223949 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":169,"thread_start_us":763,"threads_started":1}
I20260806 22:30:49.224658 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.215668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:39605 (local address 127.1.8.66:40185)
0806 22:30:49.215887 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.215894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.216848 (+ 954us) server_negotiation.cc:407] Connection header received
0806 22:30:49.216999 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.217004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.217087 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.217190 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:30:49.218002 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.219137 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.222067 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.223323 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.223456 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.223555 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.224211 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.224335 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:30:49.225234 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.215623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40185 (local address 127.1.8.65:39605)
0806 22:30:49.216744 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0806 22:30:49.216760 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:30:49.216854 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:30:49.217230 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:30:49.217235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:30:49.217326 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:30:49.217841 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.217854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.219276 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.219282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.221905 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:30:49.221920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.223781 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.223788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:30:49.223971 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.224009 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:30:49.224825 (+ 816us) client_negotiation.cc:240] Negotiation successful
0806 22:30:49.224964 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":177,"spinlock_wait_cycles":20352,"thread_start_us":535,"threads_started":1}
I20260806 22:30:49.225543 26283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8888c0f7f0412f8e32926761e49aa7" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b854f41d04c8c98b0af720ef28278" is_pre_election: true
I20260806 22:30:49.226181 26283 raft_consensus.cc:2454] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 0.
I20260806 22:30:49.226217 26086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8888c0f7f0412f8e32926761e49aa7" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f570ec1b3854ff585185f7e49cc96f4" is_pre_election: true
I20260806 22:30:49.226909 26086 raft_consensus.cc:2454] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 0.
I20260806 22:30:49.226966 25784 leader_election.cc:302] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b723a166b064107a91f1763689a105c, cf7b854f41d04c8c98b0af720ef28278; no voters:
I20260806 22:30:49.227423 26389 raft_consensus.cc:2781] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:30:49.227586 26389 raft_consensus.cc:482] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:30:49.227741 26389 raft_consensus.cc:3037] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:49.230053 26389 raft_consensus.cc:504] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.230705 26389 leader_election.cc:288] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 election: Requested vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:30:49.231171 26086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8888c0f7f0412f8e32926761e49aa7" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f570ec1b3854ff585185f7e49cc96f4"
I20260806 22:30:49.231236 26283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8888c0f7f0412f8e32926761e49aa7" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b854f41d04c8c98b0af720ef28278"
I20260806 22:30:49.231405 26086 raft_consensus.cc:3037] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:49.231423 26283 raft_consensus.cc:3037] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:30:49.234652 26283 raft_consensus.cc:2454] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 1.
I20260806 22:30:49.234652 26086 raft_consensus.cc:2454] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 1.
I20260806 22:30:49.235118 25784 leader_election.cc:302] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b723a166b064107a91f1763689a105c, cf7b854f41d04c8c98b0af720ef28278; no voters:
I20260806 22:30:49.235427 26389 raft_consensus.cc:2781] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:30:49.235882 26389 raft_consensus.cc:686] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 1 LEADER]: Becoming Leader. State: Replica: 6b723a166b064107a91f1763689a105c, State: Running, Role: LEADER
I20260806 22:30:49.236382 26389 consensus_queue.cc:237] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } }
I20260806 22:30:49.241077 25654 catalog_manager.cc:5495] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b723a166b064107a91f1763689a105c (127.1.8.65). New cstate: current_term: 1 leader_uuid: "6b723a166b064107a91f1763689a105c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } health_report { overall_health: UNKNOWN } } }
I20260806 22:30:49.282279 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.276721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.1.8.65:37403)
0806 22:30:49.276952 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.276960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.277039 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:30:49.277291 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.277295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.277376 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.277536 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:30:49.277907 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.278636 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.280130 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.280529 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.280672 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.281202 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:30:49.281686 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:30:49.281855 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.281912 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.281994 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:30:49.290257 26086 raft_consensus.cc:1261] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Refusing update from remote peer 6b723a166b064107a91f1763689a105c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:49.290362 26283 raft_consensus.cc:1261] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Refusing update from remote peer 6b723a166b064107a91f1763689a105c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:30:49.291240 26395 consensus_queue.cc:1035] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:49.291589 26389 consensus_queue.cc:1035] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:30:49.302609 26402 mvcc.cc:205] Tried to move back new op lower bound from 7315683120283070464 to 7315683120076177408. Current Snapshot: MvccSnapshot[applied={T|T < 7315683120283070464}]
I20260806 22:30:49.315380 25654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:49.315680 25654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:30:49.319416 25654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Sending DeleteTablet for 3 replicas of tablet 4e8888c0f7f0412f8e32926761e49aa7
I20260806 22:30:49.320763 26056 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:49 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:53610
I20260806 22:30:49.320763 26253 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:49 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:56032
I20260806 22:30:49.321036 25851 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8888c0f7f0412f8e32926761e49aa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:30:49 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:45654
I20260806 22:30:49.321656 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 25762
I20260806 22:30:49.337392 26423 tablet_replica.cc:330] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: stopping tablet replica
I20260806 22:30:49.337544 26419 tablet_replica.cc:330] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: stopping tablet replica
I20260806 22:30:49.338285 26417 tablet_replica.cc:330] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: stopping tablet replica
I20260806 22:30:49.338613 26423 raft_consensus.cc:2229] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:30:49.338829 26419 raft_consensus.cc:2229] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:49.339404 26417 raft_consensus.cc:2229] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:30:49.339581 26423 raft_consensus.cc:2258] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:49.340508 26417 raft_consensus.cc:2258] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:49.340821 26419 raft_consensus.cc:2258] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:30:49.341765 26423 ts_tablet_manager.cc:1905] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:49.344658 26417 ts_tablet_manager.cc:1905] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:30:49.345402 26419 ts_tablet_manager.cc:1905] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:30:49 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:30:49.351146 26423 ts_tablet_manager.cc:1918] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:49.351471 26423 log.cc:1196] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/4e8888c0f7f0412f8e32926761e49aa7
I20260806 22:30:49.352748 26419 ts_tablet_manager.cc:1918] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:49.352988 26419 log.cc:1196] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/4e8888c0f7f0412f8e32926761e49aa7
I20260806 22:30:49.353533 26419 ts_tablet_manager.cc:1939] T 4e8888c0f7f0412f8e32926761e49aa7 P cf7b854f41d04c8c98b0af720ef28278: Deleting consensus metadata
I20260806 22:30:49.353585 26423 ts_tablet_manager.cc:1939] T 4e8888c0f7f0412f8e32926761e49aa7 P 6b723a166b064107a91f1763689a105c: Deleting consensus metadata
I20260806 22:30:49.355495 25631 catalog_manager.cc:4841] TS cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): tablet 4e8888c0f7f0412f8e32926761e49aa7 (table test-table [id=45e55dc6a0f14943bcf6f962618fdac4]) successfully deleted
I20260806 22:30:49.355880 25631 catalog_manager.cc:4841] TS 6b723a166b064107a91f1763689a105c (127.1.8.65:37403): tablet 4e8888c0f7f0412f8e32926761e49aa7 (table test-table [id=45e55dc6a0f14943bcf6f962618fdac4]) successfully deleted
I20260806 22:30:49.357249 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.323191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.1.8.65:37403)
0806 22:30:49.323424 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.323433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.330224 (+ 6791us) server_negotiation.cc:407] Connection header received
0806 22:30:49.331315 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.331324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.331434 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.331533 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:49.337847 (+ 6314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.339081 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.341690 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.342367 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.342481 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.348678 (+ 6197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:49.348693 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:49.348715 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:49.348767 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:49.353446 (+ 4679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.354389 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.354396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.354399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.354612 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.355146 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.355153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.355156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.355680 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:49.355909 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.356531 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.356780 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:30:49.357998 25943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:49.358819 26417 ts_tablet_manager.cc:1918] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:30:49.359001 26417 log.cc:1196] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/4e8888c0f7f0412f8e32926761e49aa7
I20260806 22:30:49.359588 26417 ts_tablet_manager.cc:1939] T 4e8888c0f7f0412f8e32926761e49aa7 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting consensus metadata
I20260806 22:30:49.361424 25630 catalog_manager.cc:4841] TS 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): tablet 4e8888c0f7f0412f8e32926761e49aa7 (table test-table [id=45e55dc6a0f14943bcf6f962618fdac4]) successfully deleted
I20260806 22:30:49.458688 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 25762
I20260806 22:30:49.486348 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 25957
Aug 06 22:30:49 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:30:49.501750 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.487323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.1.8.66:40185)
0806 22:30:49.487498 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.487503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.487521 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:30:49.487597 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.487601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.487671 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.487754 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:49.489121 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.489899 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.491195 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.491585 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.491673 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.496050 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:49.496066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:49.496081 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:49.496171 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:49.499130 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.499767 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.499774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.499776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.499887 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.500290 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.500293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.500295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.500679 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:49.500905 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.501191 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.501351 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:30:49.502460 26139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:49.593456 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 25957
I20260806 22:30:49.617924 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 26160
Aug 06 22:30:49 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:30:49.636888 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:30:49.618735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.1.8.67:42329)
0806 22:30:49.618944 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:30:49.618950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:30:49.618971 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:30:49.619152 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:30:49.619158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:30:49.619249 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:30:49.619358 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:30:49.620876 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.621604 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.623006 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:30:49.623419 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:30:49.623562 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:30:49.629749 (+ 6187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:30:49.629776 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:30:49.629804 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:30:49.629861 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:30:49.633981 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.634906 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.634915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.634917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.635086 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:30:49.635430 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:30:49.635434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:30:49.635435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:30:49.635784 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:30:49.636015 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:30:49.636327 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:30:49.636505 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:30:49.637554 26340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:49.727624 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 26160
I20260806 22:30:49.750869 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 25608
I20260806 22:30:49.752054 25711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:30:49.866096 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 25608
I20260806 22:31:09.895749 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--webserver_interface=127.1.8.126
--webserver_port=42625
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:10.249903 26431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:10.250268 26431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:10.250448 26431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:10.257745 26431 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:31:10.257879 26431 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:31:10.257972 26431 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:31:10.258070 26431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:31:10.258173 26431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:10.258294 26431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:10.258383 26431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:31:10.258463 26431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:31:10.268502 26431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=42625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:10.269860 26431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:10.271224 26431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:10.299386 26441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:10.299554 26437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:10.299237 26438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:10.300359 26431 server_base.cc:1034] running on GCE node
I20260806 22:31:10.301344 26431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:10.303045 26431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:10.304294 26431 hybrid_clock.cc:648] HybridClock initialized: now 1786055470304211 us; error 56 us; skew 500 ppm
I20260806 22:31:10.304708 26431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:10 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:10.308772 26431 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:31:10.310614 26431 webserver.cc:460] Webserver started at http://127.1.8.126:42625/ using document root <none> and password file <none>
I20260806 22:31:10.311508 26431 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:10.311682 26431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:10.318405 26431 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:31:10.321513 26454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:10.322896 26431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:31:10.323130 26431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "69ebfbc4ca564dad9333f1b0b3853923"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "cce53dc66d784225f7f542bfc5b90d05"
server_key_iv: "ee023e422dcc00d151dd344326f7bf0c"
server_key_version: "encryptionkey@0"
I20260806 22:31:10.324285 26431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:10.356607 26431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:10.361068 26431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:10.361392 26431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:10.381661 26431 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42857
I20260806 22:31:10.381807 26524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42857 every 8 connection(s)
I20260806 22:31:10.383330 26431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:31:10.389441 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 26431
I20260806 22:31:10.390288 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:37403
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=42847
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:31:10.390915 26525 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:31:10.394393 26525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap starting.
I20260806 22:31:10.414857 26525 log.cc:824] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:10.427345 26525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:31:10.427982 26525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap complete.
I20260806 22:31:10.437813 26525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:10.438745 26525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Initialized, Role: FOLLOWER
I20260806 22:31:10.439350 26525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:10.439637 26525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:31:10.439776 26525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:31:10.439966 26525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:31:10.443869 26525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:10.444468 26525 leader_election.cc:302] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ebfbc4ca564dad9333f1b0b3853923; no voters:
I20260806 22:31:10.445192 26525 leader_election.cc:288] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:31:10.445330 26530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:31:10.446585 26530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 LEADER]: Becoming Leader. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Running, Role: LEADER
I20260806 22:31:10.447232 26530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:10.447656 26525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:31:10.449950 26532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:31:10.450435 26532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:10.450874 26533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ebfbc4ca564dad9333f1b0b3853923. Latest consensus state: current_term: 2 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:31:10.451392 26533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:10.461583 26537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:31:10.470788 26537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:31:10.472252 26537 catalog_manager.cc:1259] Loaded cluster ID: ec4b1d788b0246d1b39f3a208bb372ac
I20260806 22:31:10.472417 26537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:31:10.476563 26537 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:31:10.478449 26537 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Loaded TSK: 0
I20260806 22:31:10.496582 26552 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:31:10.519692 26537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Generated new TSK 1
I20260806 22:31:10.520145 26537 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:31:10.764375 26527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:10.764652 26527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:10.764806 26527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:10.772051 26527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:10.772244 26527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:10.772467 26527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:31:10.782598 26527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:37403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=42847
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:10.783959 26527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:10.785580 26527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:10.800606 26564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:10.800899 26563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:10.801154 26567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:10.805655 26527 server_base.cc:1034] running on GCE node
I20260806 22:31:10.806278 26527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:10.807307 26527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:10.808522 26527 hybrid_clock.cc:648] HybridClock initialized: now 1786055470808468 us; error 52 us; skew 500 ppm
I20260806 22:31:10.808861 26527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:10 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:10.812278 26527 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:31:10.813927 26527 webserver.cc:460] Webserver started at http://127.1.8.65:42847/ using document root <none> and password file <none>
I20260806 22:31:10.814680 26527 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:10.814844 26527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:10.820608 26527 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:31:10.823568 26579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:10.825192 26527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:10.825371 26527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "6b723a166b064107a91f1763689a105c"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "2c863e457fc14d552aa49684053a2b2d"
server_key_iv: "ddf741eca077e2018edd09555eb96b40"
server_key_version: "encryptionkey@0"
I20260806 22:31:10.826377 26527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:10.848976 26527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:10.853197 26527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:10.853479 26527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:10.854471 26527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:10.855989 26527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:10.856138 26527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:10.856283 26527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:10.856376 26527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:10.890211 26527 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:37403
I20260806 22:31:10.890296 26756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:37403 every 8 connection(s)
I20260806 22:31:10.891649 26527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:31:10.893129 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 26527
I20260806 22:31:10.893991 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:40185
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=40085
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:31:10 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:10.920046 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:10.895089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:55361 (local address 127.1.8.126:42857)
0806 22:31:10.895819 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:10.895838 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:10.896927 (+ 1089us) server_negotiation.cc:407] Connection header received
0806 22:31:10.900044 (+ 3117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:10.900056 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:10.900445 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:10.900731 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:10.902765 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:10.903964 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:10.906504 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:10.907149 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:10.907280 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:10.911194 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:10.911212 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:10.911236 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:10.911315 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:10.915486 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:10.916237 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:10.916244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:10.916249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:10.916416 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:10.916906 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:10.916910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:10.916913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:10.917373 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:10.917610 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:10.918338 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 22:31:10.918711 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":327,"threads_started":1}
I20260806 22:31:10.920032 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:10.895274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:55361)
0806 22:31:10.896719 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:10.896778 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:10.899485 (+ 2707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:10.900951 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:10.900963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:10.901766 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:10.902563 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:10.902590 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:10.904159 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:10.904168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:10.906315 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:10.906333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:10.907397 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:10.907407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:10.907708 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:10.908448 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:10.908475 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:10.910872 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:10.915623 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:10.915630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:10.915641 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:10.916035 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:10.916577 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:10.916583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:10.916591 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:10.916772 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:10.917522 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:10.917530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:10.917870 (+ 340us) client_negotiation.cc:769] Sending connection context
0806 22:31:10.918126 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 22:31:10.918656 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"thread_start_us":855,"threads_started":1}
I20260806 22:31:10.923182 26759 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:31:10.923568 26759 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:10.924603 26759 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:31:10.927213 26476 ts_manager.cc:194] Registered new tserver with Master: 6b723a166b064107a91f1763689a105c (127.1.8.65:37403)
I20260806 22:31:10.929049 26476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:55361
W20260806 22:31:11.239555 26767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:11.239833 26767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:11.240003 26767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:11.246934 26767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:11.247046 26767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:11.247216 26767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:31:11.256357 26767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:40185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=40085
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:11.257591 26767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:11.258864 26767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:11.272004 26777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:11.272064 26775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:11.271967 26774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:11.272868 26767 server_base.cc:1034] running on GCE node
I20260806 22:31:11.273456 26767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:11.274391 26767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:11.275625 26767 hybrid_clock.cc:648] HybridClock initialized: now 1786055471275589 us; error 50 us; skew 500 ppm
I20260806 22:31:11.275936 26767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:11 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:11.279064 26767 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:31:11.280581 26767 webserver.cc:460] Webserver started at http://127.1.8.66:40085/ using document root <none> and password file <none>
I20260806 22:31:11.281240 26767 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:11.281358 26767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:11.286808 26767 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:31:11.289675 26787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.290966 26767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:31:11.291193 26767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "0f570ec1b3854ff585185f7e49cc96f4"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "b0d7603e97aa966654fa4ef2a6fcb3c3"
server_key_iv: "ab00765434a408a982f4d7c1e5a37a73"
server_key_version: "encryptionkey@0"
I20260806 22:31:11.292167 26767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:11.316598 26767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:11.320591 26767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:11.320822 26767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:11.321767 26767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:11.323177 26767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:11.323266 26767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.323336 26767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:11.323390 26767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.358031 26767 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40185
I20260806 22:31:11.358127 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40185 every 8 connection(s)
I20260806 22:31:11.359563 26767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:31:11.359972 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 26767
I20260806 22:31:11.360857 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:42329
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=37869
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:31:11 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:11.379858 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.362476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:52867 (local address 127.1.8.126:42857)
0806 22:31:11.362668 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.362673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.364122 (+ 1449us) server_negotiation.cc:407] Connection header received
0806 22:31:11.365947 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.365953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.366051 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.366171 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:11.367670 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.368434 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.369969 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.370409 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.370546 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.374099 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:11.374104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:11.374107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:11.374140 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:11.376768 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.377503 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.377509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.377512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.377643 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.378084 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.378089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.378093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.378365 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:11.378552 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.379254 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.379487 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:31:11.380924 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.363168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:52867)
0806 22:31:11.363937 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:11.363974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:11.365670 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:11.366417 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:11.366424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:11.366947 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:11.367491 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.367509 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.368614 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.368618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.369808 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.369821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.370653 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.370661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.370890 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.371608 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:11.371630 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:11.373861 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:11.376914 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:11.376923 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:11.376939 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:11.377365 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:11.377758 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:11.377763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:11.377767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:11.377958 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:11.378512 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:11.378521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:11.378894 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 22:31:11.379169 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 22:31:11.379728 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":317,"threads_started":1}
I20260806 22:31:11.382642 26960 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:31:11.382968 26960 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:11.383831 26960 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:31:11.385627 26476 ts_manager.cc:194] Registered new tserver with Master: 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185)
I20260806 22:31:11.386986 26476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:52867
W20260806 22:31:11.710804 26966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:11.711077 26966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:11.711171 26966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:11.718252 26966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:11.718353 26966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:11.718468 26966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:31:11.727850 26966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:42329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=37869
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:11.729210 26966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:11.730552 26966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:11.743741 26977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:11.743783 26976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:11.743935 26981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:11.744652 26966 server_base.cc:1034] running on GCE node
I20260806 22:31:11.745157 26966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:11.745985 26966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:11.747207 26966 hybrid_clock.cc:648] HybridClock initialized: now 1786055471747174 us; error 28 us; skew 500 ppm
I20260806 22:31:11.747529 26966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:11 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:11.750784 26966 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:31:11.752182 26966 webserver.cc:460] Webserver started at http://127.1.8.67:37869/ using document root <none> and password file <none>
I20260806 22:31:11.752843 26966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:11.752991 26966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:11.758220 26966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 22:31:11.761010 26991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.762365 26966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:31:11.762552 26966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "cf7b854f41d04c8c98b0af720ef28278"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "2657278a9763f894f1a7591ed92cac2d"
server_key_iv: "564b568d2aebf48de03b883d861cc8e0"
server_key_version: "encryptionkey@0"
I20260806 22:31:11.763584 26966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:11.780580 26966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:11.785153 26966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:11.785435 26966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:11.786566 26966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:11.788112 26966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:11.788241 26966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.788370 26966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:11.788466 26966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:11.822764 26966 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:42329
I20260806 22:31:11.822868 27164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:42329 every 8 connection(s)
I20260806 22:31:11.824265 26966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:31:11.826524 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 26966
Aug 06 22:31:11 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:11.844349 27173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.828145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.1.8.126:42857)
0806 22:31:11.828662 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.828668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.828689 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:31:11.828764 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.828773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.828868 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.829002 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:11.830928 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.832153 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.834011 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.834660 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.834787 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.837914 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:11.837922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:11.837928 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:11.837966 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:11.841333 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.842188 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.842195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.842199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.842323 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.842748 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.842753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.842757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.843041 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:11.843340 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.843643 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.843862 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":391,"thread_start_us":842,"threads_started":1}
I20260806 22:31:11.846814 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.827249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:47339 (local address 127.1.8.126:42857)
0806 22:31:11.827505 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.827512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.828754 (+ 1242us) server_negotiation.cc:407] Connection header received
0806 22:31:11.830705 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.830711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.830792 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.830908 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:11.832722 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.833825 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.835904 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.836611 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.836761 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.840474 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:11.840478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:11.840481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:11.840519 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:11.843415 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.844209 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.844214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.844217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.844367 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.844896 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.844901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.844904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.845356 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:11.845505 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.846291 (+ 786us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.846489 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":80}
I20260806 22:31:11.847656 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.827703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:47339)
0806 22:31:11.828556 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:11.828652 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:11.830453 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:11.831185 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:11.831196 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:11.831876 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:11.832567 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.832583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.834013 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.834018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.835764 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.835776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.836809 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.836813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.837003 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.837617 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:11.837639 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:11.840266 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:11.843560 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:11.843571 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:11.843585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:11.843977 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:11.844511 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:11.844518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:11.844524 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:11.844763 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:11.845500 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:11.845507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:11.845901 (+ 394us) client_negotiation.cc:769] Sending connection context
0806 22:31:11.846128 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 22:31:11.846480 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":503,"threads_started":1}
I20260806 22:31:11.849107 27169 heartbeater.cc:347] Connected to a master server at 127.1.8.126:42857
I20260806 22:31:11.849370 27169 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:11.850032 27169 heartbeater.cc:511] Master 127.1.8.126:42857 requested a full tablet report, sending...
I20260806 22:31:11.851337 26476 ts_manager.cc:194] Registered new tserver with Master: cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:31:11.852152 26476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:47339
I20260806 22:31:11.859141 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:31:11.866061 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.860030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.1.8.126:42857)
0806 22:31:11.860228 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.860233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.860725 (+ 492us) server_negotiation.cc:407] Connection header received
0806 22:31:11.860957 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.860962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.861113 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.861263 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:31:11.861730 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.862602 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.864177 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.864604 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.864743 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.865225 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:31:11.865421 (+ 196us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:31:11.865805 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42746: token expired
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 22:31:11.866183 26765 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42746: token expired
W20260806 22:31:11.866698 26360 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.8.126:42857)
I20260806 22:31:11.878890 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.867369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.1.8.126:42857)
0806 22:31:11.867517 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.867521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.867536 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:31:11.867640 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.867643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.867693 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.867767 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:11.869135 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.869811 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.871068 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.871445 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.871531 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.874110 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:11.874114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:11.874117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:11.874145 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:11.876774 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.877315 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.877319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.877321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.877428 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:11.877767 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:11.877770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:11.877772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:11.878071 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:11.878246 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.878512 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.878661 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:31:11.882289 26476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:31:11.889669 26476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:31:11.914590 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.901805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.1.8.66:40185)
0806 22:31:11.902517 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.902529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.903477 (+ 948us) server_negotiation.cc:407] Connection header received
0806 22:31:11.903747 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.903754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.904004 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.905162 (+ 1158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:11.905480 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.907516 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.911502 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.913274 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.913510 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.913728 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.914096 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.914261 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":330,"threads_started":1}
I20260806 22:31:11.914644 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.901682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40185 (local address 127.0.0.1:41504)
0806 22:31:11.903311 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:11.903347 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:11.903530 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:11.904295 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:11.904304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:11.904340 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:11.904881 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.904899 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.907714 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.907723 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.911265 (+ 3542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.911286 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.913645 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.913652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.913862 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.913898 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:31:11.914042 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 22:31:11.914226 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":129,"thread_start_us":2597,"threads_started":1}
I20260806 22:31:11.915962 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.902486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.1.8.67:42329)
0806 22:31:11.903317 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.903328 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.904476 (+ 1148us) server_negotiation.cc:407] Connection header received
0806 22:31:11.904591 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.904601 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.904848 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.905066 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:11.906170 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.908368 (+ 2198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.911791 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.913252 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.913403 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.913660 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.915472 (+ 1812us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.915636 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":421,"threads_started":1}
I20260806 22:31:11.916854 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.901800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42329 (local address 127.0.0.1:42970)
0806 22:31:11.904159 (+ 2359us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:11.904181 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:11.904325 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:11.905396 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:11.905404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:11.905434 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:11.905931 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.905945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.908601 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.908608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.911590 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.911608 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.914840 (+ 3232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.914848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.915007 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.915038 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:31:11.915138 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:31:11.915289 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2176,"mutex_wait_us":5,"thread_start_us":1293,"threads_started":1}
I20260806 22:31:11.918941 26864 tablet_service.cc:1467] Processing CreateTablet for tablet 16eee89100ed472caaf0aae087d817c0 (DEFAULT_TABLE table=test-table [id=e0369d9db9904de39642a33a81fd59b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:11.920410 26864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16eee89100ed472caaf0aae087d817c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:11.920701 27064 tablet_service.cc:1467] Processing CreateTablet for tablet 16eee89100ed472caaf0aae087d817c0 (DEFAULT_TABLE table=test-table [id=e0369d9db9904de39642a33a81fd59b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:11.921450 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.903798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55466)
0806 22:31:11.912188 (+ 8390us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:11.912206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:11.912328 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:11.912995 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:11.913000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:11.913027 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:11.913525 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.913538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.915150 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.915158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.917747 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:11.917760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.920342 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.920349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:11.920786 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.920819 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:31:11.920925 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:31:11.921051 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8209,"thread_start_us":8086,"threads_started":1}
I20260806 22:31:11.921499 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:11.904095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.1.8.65:37403)
0806 22:31:11.911866 (+ 7771us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:11.911876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:11.912445 (+ 569us) server_negotiation.cc:407] Connection header received
0806 22:31:11.912568 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:11.912575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:11.912791 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:11.912995 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:11.913730 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.914705 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.917899 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:11.919394 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:11.919551 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:11.919792 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:11.921003 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0806 22:31:11.921169 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7582,"thread_start_us":7450,"threads_started":1}
I20260806 22:31:11.922053 27064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16eee89100ed472caaf0aae087d817c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:11.925271 26653 tablet_service.cc:1467] Processing CreateTablet for tablet 16eee89100ed472caaf0aae087d817c0 (DEFAULT_TABLE table=test-table [id=e0369d9db9904de39642a33a81fd59b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:11.926644 26653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16eee89100ed472caaf0aae087d817c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:11.932753 26759 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:31:11.933409 27196 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Bootstrap starting.
I20260806 22:31:11.937419 27196 tablet_bootstrap.cc:654] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:11.938638 27196 log.cc:824] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:11.940085 27197 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Bootstrap starting.
I20260806 22:31:11.941926 27196 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: No bootstrap required, opened a new log
I20260806 22:31:11.942214 27196 ts_tablet_manager.cc:1397] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:31:11.943648 27197 tablet_bootstrap.cc:654] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:11.944034 27200 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Bootstrap starting.
I20260806 22:31:11.944912 27197 log.cc:824] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:11.947196 27200 tablet_bootstrap.cc:654] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:11.948024 27197 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: No bootstrap required, opened a new log
I20260806 22:31:11.948232 27200 log.cc:824] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:11.948345 27197 ts_tablet_manager.cc:1397] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260806 22:31:11.950697 27200 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: No bootstrap required, opened a new log
I20260806 22:31:11.951213 27200 ts_tablet_manager.cc:1397] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 22:31:11.952713 27196 raft_consensus.cc:348] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.953159 27196 raft_consensus.cc:374] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:11.953256 27196 raft_consensus.cc:729] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f570ec1b3854ff585185f7e49cc96f4, State: Initialized, Role: FOLLOWER
I20260806 22:31:11.953866 27196 consensus_queue.cc:260] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.954998 26960 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:31:11.955629 27196 ts_tablet_manager.cc:1428] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260806 22:31:11.958001 27197 raft_consensus.cc:348] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.958345 27197 raft_consensus.cc:374] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:11.958467 27197 raft_consensus.cc:729] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7b854f41d04c8c98b0af720ef28278, State: Initialized, Role: FOLLOWER
I20260806 22:31:11.959089 27197 consensus_queue.cc:260] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.960230 27169 heartbeater.cc:503] Master 127.1.8.126:42857 was elected leader, sending a full tablet report...
I20260806 22:31:11.960878 27197 ts_tablet_manager.cc:1428] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260806 22:31:11.962395 27200 raft_consensus.cc:348] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.962874 27200 raft_consensus.cc:374] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:11.962990 27200 raft_consensus.cc:729] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b723a166b064107a91f1763689a105c, State: Initialized, Role: FOLLOWER
I20260806 22:31:11.963569 27200 consensus_queue.cc:260] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:11.965308 27200 ts_tablet_manager.cc:1428] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Time spent starting tablet: real 0.014s user 0.003s sys 0.008s
W20260806 22:31:12.076009 27170 tablet.cc:2369] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:31:12.079335 27208 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:12.079588 27208 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:12.081379 27208 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:31:12.088567 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.081727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:55151 (local address 127.1.8.66:40185)
0806 22:31:12.081972 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.081979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.082928 (+ 949us) server_negotiation.cc:407] Connection header received
0806 22:31:12.083167 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.083171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.083233 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.083333 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:12.083893 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.084684 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.086580 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.087620 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.087766 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.087868 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.088188 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.088311 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:31:12.088773 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.082112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:56319 (local address 127.1.8.67:42329)
0806 22:31:12.082302 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.082308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.083237 (+ 929us) server_negotiation.cc:407] Connection header received
0806 22:31:12.083441 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.083445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.083501 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.083605 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:12.084170 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.084895 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.086753 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.087812 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.087956 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.088066 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.088387 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.088512 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:31:12.088862 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.081623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40185 (local address 127.1.8.65:55151)
0806 22:31:12.082803 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.082819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:12.083008 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:12.083350 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:12.083355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:12.083381 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:12.083732 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:12.083740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.084818 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.084824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:12.086426 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:12.086439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.087837 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.087843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:12.087997 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.088024 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 22:31:12.088156 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 22:31:12.088281 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":754,"threads_started":1}
I20260806 22:31:12.089247 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.082378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42329 (local address 127.1.8.65:56319)
0806 22:31:12.083135 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.083149 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:12.083303 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:12.083626 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:12.083631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:12.083659 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:12.084001 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:12.084009 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.085042 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.085048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:12.086601 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:12.086613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.088058 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.088065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:12.088230 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.088254 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:31:12.088360 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:31:12.088486 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":1,"thread_start_us":476,"threads_started":1}
I20260806 22:31:12.090054 26898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16eee89100ed472caaf0aae087d817c0" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f570ec1b3854ff585185f7e49cc96f4" is_pre_election: true
I20260806 22:31:12.090247 27093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16eee89100ed472caaf0aae087d817c0" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b854f41d04c8c98b0af720ef28278" is_pre_election: true
I20260806 22:31:12.090672 26898 raft_consensus.cc:2454] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 0.
I20260806 22:31:12.090742 27093 raft_consensus.cc:2454] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 0.
I20260806 22:31:12.091532 26580 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f570ec1b3854ff585185f7e49cc96f4, 6b723a166b064107a91f1763689a105c; no voters:
I20260806 22:31:12.091958 27208 raft_consensus.cc:2781] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:31:12.092124 27208 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:31:12.092237 27208 raft_consensus.cc:3037] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:12.094576 27208 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:12.095280 27208 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 election: Requested vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
I20260806 22:31:12.095680 26898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16eee89100ed472caaf0aae087d817c0" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f570ec1b3854ff585185f7e49cc96f4"
I20260806 22:31:12.095829 27093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16eee89100ed472caaf0aae087d817c0" candidate_uuid: "6b723a166b064107a91f1763689a105c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b854f41d04c8c98b0af720ef28278"
I20260806 22:31:12.095911 26898 raft_consensus.cc:3037] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:12.096057 27093 raft_consensus.cc:3037] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:12.099300 26898 raft_consensus.cc:2454] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 1.
I20260806 22:31:12.099298 27093 raft_consensus.cc:2454] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b723a166b064107a91f1763689a105c in term 1.
I20260806 22:31:12.099730 26582 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f570ec1b3854ff585185f7e49cc96f4, 6b723a166b064107a91f1763689a105c, cf7b854f41d04c8c98b0af720ef28278; no voters:
I20260806 22:31:12.100136 27208 raft_consensus.cc:2781] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:31:12.100615 27208 raft_consensus.cc:686] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 LEADER]: Becoming Leader. State: Replica: 6b723a166b064107a91f1763689a105c, State: Running, Role: LEADER
I20260806 22:31:12.101152 27208 consensus_queue.cc:237] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:12.105470 26476 catalog_manager.cc:5495] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b723a166b064107a91f1763689a105c (127.1.8.65). New cstate: current_term: 1 leader_uuid: "6b723a166b064107a91f1763689a105c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } health_report { overall_health: HEALTHY } } }
W20260806 22:31:12.111898 26962 tablet.cc:2369] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:31:12.144559 26761 tablet.cc:2369] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:31:12.159312 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.153861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.1.8.65:37403)
0806 22:31:12.154086 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.154092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.154111 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:31:12.154325 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.154329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.154410 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.154572 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:31:12.154962 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.155718 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.157160 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.157569 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.157706 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.158231 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:31:12.158718 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:31:12.158882 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.158964 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.159046 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:31:12.167573 27093 raft_consensus.cc:1261] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Refusing update from remote peer 6b723a166b064107a91f1763689a105c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:12.167590 26898 raft_consensus.cc:1261] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Refusing update from remote peer 6b723a166b064107a91f1763689a105c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:12.168588 27212 consensus_queue.cc:1035] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:31:12.168962 27208 consensus_queue.cc:1035] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:31:12.179895 27215 mvcc.cc:205] Tried to move back new op lower bound from 7315683213987946496 to 7315683213730193408. Current Snapshot: MvccSnapshot[applied={T|T < 7315683213987946496}]
I20260806 22:31:12.181275 27216 mvcc.cc:205] Tried to move back new op lower bound from 7315683213987946496 to 7315683213730193408. Current Snapshot: MvccSnapshot[applied={T|T < 7315683213987946496}]
I20260806 22:31:12.193457 26474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:12.193812 26474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:12.197708 26474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Sending DeleteTablet for 3 replicas of tablet 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:12.198767 26864 tablet_service.cc:1514] Processing DeleteTablet for tablet 16eee89100ed472caaf0aae087d817c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:12 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:41504
I20260806 22:31:12.199021 27064 tablet_service.cc:1514] Processing DeleteTablet for tablet 16eee89100ed472caaf0aae087d817c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:12 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:42970
I20260806 22:31:12.199227 26653 tablet_service.cc:1514] Processing DeleteTablet for tablet 16eee89100ed472caaf0aae087d817c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:12 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:55466
I20260806 22:31:12.200357 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 26527
I20260806 22:31:12.230979 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.201462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.1.8.65:37403)
0806 22:31:12.201609 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.201615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.201634 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:31:12.201819 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.201828 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.201935 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.202045 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:12.204665 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.206342 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.209081 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.210120 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.210269 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.218461 (+ 8192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:12.218480 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:12.218504 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:12.218586 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:12.224032 (+ 5446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.225261 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.225274 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.225280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.225535 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.227058 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.227067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.227073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.227666 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:12.228047 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.230121 (+ 2074us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.230462 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:31:12.231922 26749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:12.242139 27235 tablet_replica.cc:330] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: stopping tablet replica
I20260806 22:31:12.242704 27235 raft_consensus.cc:2229] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:12.243254 27235 raft_consensus.cc:2258] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:12.246078 27235 ts_tablet_manager.cc:1905] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:12.247092 27232 tablet_replica.cc:330] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: stopping tablet replica
I20260806 22:31:12.247502 27232 raft_consensus.cc:2229] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:12.247984 27232 raft_consensus.cc:2258] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:12.249895 27232 ts_tablet_manager.cc:1905] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:12.253322 27235 ts_tablet_manager.cc:1918] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:12.253490 27235 log.cc:1196] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/16eee89100ed472caaf0aae087d817c0
I20260806 22:31:12.254143 27235 ts_tablet_manager.cc:1939] T 16eee89100ed472caaf0aae087d817c0 P cf7b854f41d04c8c98b0af720ef28278: Deleting consensus metadata
I20260806 22:31:12.256036 26456 catalog_manager.cc:4841] TS cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): tablet 16eee89100ed472caaf0aae087d817c0 (table test-table [id=e0369d9db9904de39642a33a81fd59b7]) successfully deleted
I20260806 22:31:12.256600 27232 ts_tablet_manager.cc:1918] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:12.256747 27232 log.cc:1196] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/16eee89100ed472caaf0aae087d817c0
I20260806 22:31:12.257190 27232 ts_tablet_manager.cc:1939] T 16eee89100ed472caaf0aae087d817c0 P 0f570ec1b3854ff585185f7e49cc96f4: Deleting consensus metadata
I20260806 22:31:12.258543 26455 catalog_manager.cc:4841] TS 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): tablet 16eee89100ed472caaf0aae087d817c0 (table test-table [id=e0369d9db9904de39642a33a81fd59b7]) successfully deleted
I20260806 22:31:12.345800 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 26527
I20260806 22:31:12.376590 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 26767
W20260806 22:31:12.376577 26456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:37403 (error 108)
I20260806 22:31:12.378253 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.377815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55480)
0806 22:31:12.378029 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.378124 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:31:12.378470 26456 catalog_manager.cc:4584] TS 6b723a166b064107a91f1763689a105c (127.1.8.65:37403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16eee89100ed472caaf0aae087d817c0: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
I20260806 22:31:12.392285 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.377511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.1.8.66:40185)
0806 22:31:12.377713 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.377718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.377755 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:31:12.377911 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.377917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.377995 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.378112 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:12.379574 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.380500 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.382409 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.382809 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.382912 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.386148 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:12.386160 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:12.386176 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:12.386207 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:12.389557 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.390313 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.390320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.390323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.390438 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.390824 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.390828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.390830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.391196 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:12.391417 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.391766 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.391921 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:31:12.393052 26955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:12.432065 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.431624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55488)
0806 22:31:12.431822 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.431923 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:31:12.433243 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.432874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55498)
0806 22:31:12.433048 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.433127 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 22:31:12.485680 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 26767
I20260806 22:31:12.494861 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.494250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55512)
0806 22:31:12.494535 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.494647 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 22:31:12.495934 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.495576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55518)
0806 22:31:12.495732 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.495813 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:31:12.515026 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 26966
I20260806 22:31:12.528787 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.515808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.1.8.67:42329)
0806 22:31:12.516007 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:12.516015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:12.516039 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:31:12.516294 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:12.516299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:12.516366 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:12.516484 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:12.517855 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.518571 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.519862 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:12.520406 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:12.520576 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:12.523121 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:12.523137 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:12.523191 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:12.523236 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:12.526118 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.526776 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.526785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.526787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.526933 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:12.527284 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:12.527287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:12.527289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:12.527621 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:12.527922 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:12.528326 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 22:31:12.528522 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:31:12.529489 27159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:12.576812 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.576274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55534)
0806 22:31:12.576517 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.576616 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 22:31:12.578006 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.577600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55536)
0806 22:31:12.577797 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.577872 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 22:31:12.620308 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 26966
I20260806 22:31:12.646481 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 26431
I20260806 22:31:12.647763 26518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:12.763537 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:12.762718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:37403 (local address 127.0.0.1:55540)
0806 22:31:12.762984 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:12.763089 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 22:31:12.763566 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 26431
I20260806 22:31:32.797829 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--webserver_interface=127.1.8.126
--webserver_port=42625
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:42857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:33.157076 27244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:33.157426 27244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:33.157631 27244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:33.165199 27244 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:31:33.165336 27244 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:31:33.165416 27244 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:31:33.165505 27244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:31:33.165593 27244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:33.165690 27244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:33.165757 27244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:31:33.165843 27244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:31:33.177057 27244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:42857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:42857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=42625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:33.178385 27244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:33.179836 27244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:33.193259 27250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:33.193323 27252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:33.193297 27255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:33.194111 27244 server_base.cc:1034] running on GCE node
I20260806 22:31:33.194998 27244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:33.196537 27244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:33.197800 27244 hybrid_clock.cc:648] HybridClock initialized: now 1786055493197743 us; error 33 us; skew 500 ppm
I20260806 22:31:33.198171 27244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:33 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:33.201620 27244 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:31:33.203233 27244 webserver.cc:460] Webserver started at http://127.1.8.126:42625/ using document root <none> and password file <none>
I20260806 22:31:33.203964 27244 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:33.204144 27244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:33.209820 27244 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:31:33.212818 27264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:33.214176 27244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:33.214368 27244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "69ebfbc4ca564dad9333f1b0b3853923"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "cce53dc66d784225f7f542bfc5b90d05"
server_key_iv: "ee023e422dcc00d151dd344326f7bf0c"
server_key_version: "encryptionkey@0"
I20260806 22:31:33.215365 27244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:33.234256 27244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:33.238801 27244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:33.239130 27244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:33.258632 27244 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:42857
I20260806 22:31:33.258775 27345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:42857 every 8 connection(s)
I20260806 22:31:33.260342 27244 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:31:33.263903 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 27244
I20260806 22:31:33.264814 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:37403
--local_ip_for_outbound_sockets=127.1.8.65
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=42847
--webserver_interface=127.1.8.65
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:31:33.267470 27346 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:31:33.270874 27346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap starting.
I20260806 22:31:33.296365 27346 log.cc:824] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:33.318957 27346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:31:33.319708 27346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Bootstrap complete.
I20260806 22:31:33.333447 27346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:33.334826 27346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Initialized, Role: FOLLOWER
I20260806 22:31:33.335575 27346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:33.335858 27346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:31:33.336009 27346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:31:33.336256 27346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:31:33.342368 27346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:33.343016 27346 leader_election.cc:302] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ebfbc4ca564dad9333f1b0b3853923; no voters:
I20260806 22:31:33.343806 27346 leader_election.cc:288] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:31:33.344003 27352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:31:33.345502 27352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [term 3 LEADER]: Becoming Leader. State: Replica: 69ebfbc4ca564dad9333f1b0b3853923, State: Running, Role: LEADER
I20260806 22:31:33.346103 27352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } }
I20260806 22:31:33.346400 27346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:31:33.348702 27356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ebfbc4ca564dad9333f1b0b3853923. Latest consensus state: current_term: 3 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:31:33.349203 27356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:33.349793 27354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "69ebfbc4ca564dad9333f1b0b3853923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ebfbc4ca564dad9333f1b0b3853923" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 42857 } } }
I20260806 22:31:33.350101 27354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:33.360631 27363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:31:33.370688 27363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:31:33.372450 27363 catalog_manager.cc:1259] Loaded cluster ID: ec4b1d788b0246d1b39f3a208bb372ac
I20260806 22:31:33.372656 27363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:31:33.377451 27363 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:31:33.379276 27363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Loaded TSK: 0
I20260806 22:31:33.379551 27363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69ebfbc4ca564dad9333f1b0b3853923: Loaded TSK: 1
F20260806 22:31:33.380932 27363 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd2a0ab79ad google::LogMessage::Fail() at ??:0
@ 0x7fd2a0aba88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd2a0ab74de google::LogMessage::Flush() at ??:0
@ 0x7fd2a0ab8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd2a481b7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd2a480a73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd2b64e61c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd2b64e5783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd2b65712a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fd2b65710c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fd2b65bb5e9 std::function<>::operator()() at ??:0
@ 0x7fd2b64e9cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd2b64e7c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd2b656f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd2b656f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd2a3703345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd2a370691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd2a3706799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd2a36e1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd2a69bb6db start_thread at ??:0
@ 0x7fd29f95161f clone at ??:0
W20260806 22:31:33.712496 27349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:33.712756 27349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:33.712945 27349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:33.720134 27349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:33.720283 27349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:33.720471 27349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:31:33.729852 27349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:37403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=42847
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:33.731104 27349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:33.732497 27349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:33.745563 27417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:33.745467 27426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:33.745543 27422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:33.746313 27349 server_base.cc:1034] running on GCE node
I20260806 22:31:33.746930 27349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:33.747825 27349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:33.749065 27349 hybrid_clock.cc:648] HybridClock initialized: now 1786055493749006 us; error 72 us; skew 500 ppm
I20260806 22:31:33.749486 27349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:33 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:33.752748 27349 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:31:33.754257 27349 webserver.cc:460] Webserver started at http://127.1.8.65:42847/ using document root <none> and password file <none>
I20260806 22:31:33.754894 27349 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:33.755003 27349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:33.760291 27349 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:33.763023 27439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:33.764353 27349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:33.764520 27349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "6b723a166b064107a91f1763689a105c"
format_stamp: "Formatted at 2026-08-06 22:30:47 on dist-test-slave-7vjv"
server_key: "2c863e457fc14d552aa49684053a2b2d"
server_key_iv: "ddf741eca077e2018edd09555eb96b40"
server_key_version: "encryptionkey@0"
I20260806 22:31:33.765453 27349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:33.788717 27349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:33.792585 27349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:33.792845 27349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:33.793788 27349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:33.795593 27450 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 22:31:33.801198 27349 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:31:33.801296 27349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 22:31:33.801451 27349 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:31:33.804557 27349 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:31:33.804636 27349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:33.805076 27450 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Bootstrap starting.
I20260806 22:31:33.835342 27450 log.cc:824] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:33.849637 27450 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:31:33.850274 27450 tablet_bootstrap.cc:492] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Bootstrap complete.
I20260806 22:31:33.851763 27450 ts_tablet_manager.cc:1397] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Time spent bootstrapping tablet: real 0.047s user 0.027s sys 0.016s
I20260806 22:31:33.852869 27349 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:37403
I20260806 22:31:33.853031 27610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:37403 every 8 connection(s)
I20260806 22:31:33.854171 27349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:31:33.859632 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:33.857446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:57237)
0806 22:31:33.858355 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:33.858635 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":408,"threads_started":1}
W20260806 22:31:33.860468 27611 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:42857 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:42857: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
I20260806 22:31:33.860950 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 27349
I20260806 22:31:33.861575 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:33.861090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:58877)
0806 22:31:33.861288 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:33.861395 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 22:31:33.861668 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:40185
--local_ip_for_outbound_sockets=127.1.8.66
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=40085
--webserver_interface=127.1.8.66
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:31:33.860707 27450 raft_consensus.cc:348] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:33.862097 27450 raft_consensus.cc:729] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b723a166b064107a91f1763689a105c, State: Initialized, Role: FOLLOWER
I20260806 22:31:33.862695 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:33.862233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:48513)
0806 22:31:33.862407 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:33.862522 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:31:33.863018 27611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:31:33.862861 27450 consensus_queue.cc:260] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:33.864666 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:33.864261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:39517)
0806 22:31:33.864430 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:33.864518 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 22:31:33.864827 27450 ts_tablet_manager.cc:1428] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c: Time spent starting tablet: real 0.013s user 0.010s sys 0.006s
I20260806 22:31:33.865967 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:33.865561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:37525)
0806 22:31:33.865727 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:33.865812 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 22:31:34.217118 27619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:34.217459 27619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:34.217633 27619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:34.224858 27619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:34.225021 27619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:34.225234 27619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:31:34.234874 27619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:40185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=40085
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:34.236336 27619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:34.237836 27619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:34.251932 27627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:34.252036 27633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:34.252027 27628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:34.252682 27619 server_base.cc:1034] running on GCE node
I20260806 22:31:34.253360 27619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:34.254392 27619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:34.255632 27619 hybrid_clock.cc:648] HybridClock initialized: now 1786055494255583 us; error 43 us; skew 500 ppm
I20260806 22:31:34.255971 27619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:34 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055494, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:34.259279 27619 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:31:34.260826 27619 webserver.cc:460] Webserver started at http://127.1.8.66:40085/ using document root <none> and password file <none>
I20260806 22:31:34.261544 27619 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:34.261710 27619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:34.267336 27619 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:31:34.270352 27647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.271713 27619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:31:34.271956 27619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "0f570ec1b3854ff585185f7e49cc96f4"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "b0d7603e97aa966654fa4ef2a6fcb3c3"
server_key_iv: "ab00765434a408a982f4d7c1e5a37a73"
server_key_version: "encryptionkey@0"
I20260806 22:31:34.273015 27619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:34.307108 27619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:34.311573 27619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:34.311875 27619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:34.312956 27619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:34.314304 27619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:34.314440 27619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.314589 27619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:34.314682 27619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.353520 27619 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40185
I20260806 22:31:34.353608 27826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40185 every 8 connection(s)
I20260806 22:31:34.355123 27619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:31:34.359911 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.358100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:56237)
0806 22:31:34.358827 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.359074 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":283,"threads_started":1}
W20260806 22:31:34.360620 27828 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:42857 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:42857: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
I20260806 22:31:34.361392 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.361092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:32977)
0806 22:31:34.361231 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.361294 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:31:34.362227 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.361902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:56859)
0806 22:31:34.362046 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.362124 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:31:34.362454 27828 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:31:34.365562 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 27619
I20260806 22:31:34.366200 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:42329
--local_ip_for_outbound_sockets=127.1.8.67
--tserver_master_addrs=127.1.8.126:42857
--webserver_port=37869
--webserver_interface=127.1.8.67
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:34.708034 27833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:34.708333 27833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:34.708524 27833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:34.715737 27833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:34.715852 27833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:34.716017 27833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:31:34.726104 27833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:42329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=37869
--tserver_master_addrs=127.1.8.126:42857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:34.727463 27833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:34.728838 27833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:34.742091 27841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:34.742643 27845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:34.742748 27843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:34.743273 27833 server_base.cc:1034] running on GCE node
I20260806 22:31:34.743944 27833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:34.745292 27833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:34.746536 27833 hybrid_clock.cc:648] HybridClock initialized: now 1786055494746465 us; error 56 us; skew 500 ppm
I20260806 22:31:34.746922 27833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:34 dist-test-slave-7vjv krb5kdc[25591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055494, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:34.751937 27833 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:31:34.753865 27833 webserver.cc:460] Webserver started at http://127.1.8.67:37869/ using document root <none> and password file <none>
I20260806 22:31:34.754786 27833 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:34.754935 27833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:34.760596 27833 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 22:31:34.763265 27856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.764663 27833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:34.764839 27833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "cf7b854f41d04c8c98b0af720ef28278"
format_stamp: "Formatted at 2026-08-06 22:30:48 on dist-test-slave-7vjv"
server_key: "2657278a9763f894f1a7591ed92cac2d"
server_key_iv: "564b568d2aebf48de03b883d861cc8e0"
server_key_version: "encryptionkey@0"
I20260806 22:31:34.765817 27833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:34.783591 27833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:34.788604 27833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:34.788882 27833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:34.789844 27833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:34.791168 27833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:34.791283 27833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.791419 27833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:34.791517 27833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:34.825047 27833 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:42329
I20260806 22:31:34.825127 28025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:42329 every 8 connection(s)
I20260806 22:31:34.826537 27833 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:31:34.828536 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 27833
I20260806 22:31:34.831751 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.829885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:54315)
0806 22:31:34.830719 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.830904 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":381,"threads_started":1}
W20260806 22:31:34.832445 28026 heartbeater.cc:644] Failed to heartbeat to 127.1.8.126:42857 (0 consecutive failures): Network error: Failed to ping master at 127.1.8.126:42857: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
I20260806 22:31:34.833228 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.832923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:41639)
0806 22:31:34.833056 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.833118 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:31:34.834010 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.833729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:43307)
0806 22:31:34.833842 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.833910 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 22:31:34.834264 28026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:31:34.867462 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:34.866720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:42403)
0806 22:31:34.867165 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:34.867261 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":208,"threads_started":1}
I20260806 22:31:35.098711 28033 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:35.099026 28033 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:35.100811 28033 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
Aug 06 22:31:35 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
Aug 06 22:31:35 dist-test-slave-7vjv krb5kdc[25591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:31:35.127442 28035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.101441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:57743 (local address 127.1.8.67:42329)
0806 22:31:35.103471 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:35.103486 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:35.103537 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:31:35.105913 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:35.105926 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:35.106460 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:35.106836 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:35.109567 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.111462 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.114322 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.115377 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.115571 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:35.120231 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:35.120240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:35.120254 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:35.120292 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:35.123458 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:35.124457 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:35.124464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:35.124469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:35.124625 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:35.125126 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:35.125130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:35.125132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:35.125636 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:35.125835 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:35.126608 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 22:31:35.126975 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1796,"thread_start_us":1591,"threads_started":1}
I20260806 22:31:35.127472 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.101454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:39255 (local address 127.1.8.66:40185)
0806 22:31:35.103644 (+ 2190us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:35.103657 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:35.103692 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:31:35.105935 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:35.105946 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:35.106460 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:35.106833 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:35.109567 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.111462 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.114322 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.115359 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.115571 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:35.120450 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:35.120457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:35.120469 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:35.120510 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:35.123455 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:35.124430 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:35.124445 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:35.124450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:35.124609 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:35.125132 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:35.125139 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:35.125146 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:35.125631 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:35.125832 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:35.126617 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 22:31:35.127050 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2019,"thread_start_us":1868,"threads_started":1}
I20260806 22:31:35.127489 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.101263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40185 (local address 127.1.8.65:39255)
0806 22:31:35.102979 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:35.102999 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:35.103123 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:35.107022 (+ 3899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:35.107034 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:35.108124 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:35.109316 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:35.109344 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.111682 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.111686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:35.114139 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:35.114155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.115606 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.115611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:35.115983 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:35.117032 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:35.117052 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:35.120187 (+ 3135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:35.123615 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:35.123625 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:35.123641 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:35.124283 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:35.124733 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:35.124739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:35.124743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:35.125001 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:35.125781 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:35.125786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:35.126109 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 22:31:35.126373 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 22:31:35.126951 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"mutex_wait_us":708,"spinlock_wait_cycles":26496,"thread_start_us":1164,"threads_started":1}
I20260806 22:31:35.127559 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.101263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42329 (local address 127.1.8.65:57743)
0806 22:31:35.101424 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:35.101480 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:35.102133 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:35.107022 (+ 4889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:35.107034 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:35.108131 (+ 1097us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:35.109315 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:35.109344 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.111677 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.111683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:35.114139 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:35.114155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:35.115595 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:35.115602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:35.115982 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:35.116947 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:35.116995 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:35.119883 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:35.123615 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:35.123625 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:35.123641 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:35.124306 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:35.124746 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:35.124751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:35.124754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:35.124964 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:35.125777 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:35.125784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:35.126110 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 22:31:35.126373 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 22:31:35.127111 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":698,"spinlock_wait_cycles":38016}
W20260806 22:31:35.130795 27440 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
W20260806 22:31:35.131044 27444 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:35.131210 27444 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b723a166b064107a91f1763689a105c; no voters: 0f570ec1b3854ff585185f7e49cc96f4, cf7b854f41d04c8c98b0af720ef28278
I20260806 22:31:35.131628 28033 raft_consensus.cc:2726] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:31:35.363711 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.362915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:43277)
0806 22:31:35.363350 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:35.363477 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":217,"threads_started":1}
I20260806 22:31:35.835438 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.834749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:59857)
0806 22:31:35.835136 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:35.835232 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":191,"threads_started":1}
I20260806 22:31:35.869062 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:35.868356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:36009)
0806 22:31:35.868793 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:35.868893 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":203,"threads_started":1}
I20260806 22:31:36.365270 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:36.364607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:41675)
0806 22:31:36.365045 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:36.365121 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":228,"threads_started":1}
I20260806 22:31:36.837322 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:36.836462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:58529)
0806 22:31:36.837001 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:36.837137 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":286,"threads_started":1}
I20260806 22:31:36.870827 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:36.869867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:41223)
0806 22:31:36.870482 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:36.870610 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":291,"threads_started":1}
I20260806 22:31:37.015013 28052 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:37.015232 28052 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:37.015940 28052 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
W20260806 22:31:37.017079 27440 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
W20260806 22:31:37.017320 27444 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:37.017434 27444 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b723a166b064107a91f1763689a105c; no voters: 0f570ec1b3854ff585185f7e49cc96f4, cf7b854f41d04c8c98b0af720ef28278
I20260806 22:31:37.017693 28052 raft_consensus.cc:2726] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:31:37.367233 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:37.366271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:50877)
0806 22:31:37.366874 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:37.366982 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":293,"threads_started":1}
I20260806 22:31:37.838902 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:37.838228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:52301)
0806 22:31:37.838660 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:37.838738 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":239,"threads_started":1}
I20260806 22:31:37.872550 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:37.871802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:48427)
0806 22:31:37.872307 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:37.872384 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":243,"threads_started":1}
I20260806 22:31:38.368875 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:38.368131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:49585)
0806 22:31:38.368572 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:38.368649 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":206,"threads_started":1}
I20260806 22:31:38.840709 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:38.839912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:42931)
0806 22:31:38.840434 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:38.840520 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":295,"threads_started":1}
I20260806 22:31:38.874421 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:38.873676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:51599)
0806 22:31:38.874143 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:38.874235 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":256,"threads_started":1}
I20260806 22:31:39.370571 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:39.369800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:42157)
0806 22:31:39.370244 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:39.370319 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
I20260806 22:31:39.612802 28068 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:39.613057 28068 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:39.613715 28068 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
W20260806 22:31:39.614789 27444 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
W20260806 22:31:39.615024 27440 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:39.615142 27440 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b723a166b064107a91f1763689a105c; no voters: 0f570ec1b3854ff585185f7e49cc96f4, cf7b854f41d04c8c98b0af720ef28278
I20260806 22:31:39.615347 28068 raft_consensus.cc:2726] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:31:39.842314 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:39.841604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:47881)
0806 22:31:39.842068 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:39.842154 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":260,"threads_started":1}
I20260806 22:31:39.876353 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:39.875420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:60331)
0806 22:31:39.875961 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:39.876066 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":290,"threads_started":1}
I20260806 22:31:40.372078 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:40.371374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:55959)
0806 22:31:40.371825 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:40.371904 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":240,"threads_started":1}
I20260806 22:31:40.844067 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:40.843314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:44253)
0806 22:31:40.843783 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:40.843864 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
I20260806 22:31:40.878060 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:40.877258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:59475)
0806 22:31:40.877724 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:40.877863 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":253,"threads_started":1}
I20260806 22:31:41.373754 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:41.372982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:44845)
0806 22:31:41.373479 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:41.373568 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":235,"threads_started":1}
I20260806 22:31:41.846300 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:41.845129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:56831)
0806 22:31:41.845978 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:41.846066 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":417,"threads_started":1}
I20260806 22:31:41.879757 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:41.878945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:33771)
0806 22:31:41.879485 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:41.879569 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":316,"threads_started":1}
I20260806 22:31:42.377002 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:42.374920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:60265)
0806 22:31:42.376036 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:42.376492 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":542,"threads_started":1}
I20260806 22:31:42.848212 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:42.847411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:45435)
0806 22:31:42.847853 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:42.847949 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
I20260806 22:31:42.881598 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:42.880749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:37803)
0806 22:31:42.881303 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:42.881392 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":325,"threads_started":1}
I20260806 22:31:43.379274 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:43.378504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:54975)
0806 22:31:43.379030 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:43.379114 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":250,"threads_started":1}
I20260806 22:31:43.602577 28091 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:43.602835 28091 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:43.603504 28091 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
W20260806 22:31:43.604660 27444 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
W20260806 22:31:43.604890 27440 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:43.605036 27440 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b723a166b064107a91f1763689a105c; no voters: 0f570ec1b3854ff585185f7e49cc96f4, cf7b854f41d04c8c98b0af720ef28278
I20260806 22:31:43.605342 28091 raft_consensus.cc:2726] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:31:43.850296 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:43.849310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:50149)
0806 22:31:43.849961 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:43.850062 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":363,"threads_started":1}
I20260806 22:31:43.884207 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:43.882671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:53699)
0806 22:31:43.883801 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:43.883909 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":318,"threads_started":1}
I20260806 22:31:44.381059 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:44.380305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:56451)
0806 22:31:44.380777 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:44.380856 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":245,"threads_started":1}
I20260806 22:31:44.852355 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:44.851421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:42647)
0806 22:31:44.851982 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:44.852070 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":300,"threads_started":1}
I20260806 22:31:44.886046 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:44.885223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:37491)
0806 22:31:44.885689 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:44.885781 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":228,"threads_started":1}
I20260806 22:31:45.382839 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:45.382082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:36835)
0806 22:31:45.382578 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:45.382676 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
I20260806 22:31:45.854151 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:45.853290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:47085)
0806 22:31:45.853832 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:45.853936 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":259,"threads_started":1}
I20260806 22:31:45.887718 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:45.886979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:58595)
0806 22:31:45.887466 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:45.887548 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":233,"threads_started":1}
I20260806 22:31:46.384676 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:46.383873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:36275)
0806 22:31:46.384404 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:46.384487 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":311,"threads_started":1}
I20260806 22:31:46.856000 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:46.855158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:58135)
0806 22:31:46.855698 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:46.855780 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":315,"threads_started":1}
I20260806 22:31:46.889731 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:46.888818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:52331)
0806 22:31:46.889376 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:46.889457 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":334,"threads_started":1}
I20260806 22:31:47.386337 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:47.385553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:43585)
0806 22:31:47.386017 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:47.386140 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":228,"threads_started":1}
I20260806 22:31:47.857641 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:47.856947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:40701)
0806 22:31:47.857393 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:47.857479 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":235,"threads_started":1}
I20260806 22:31:47.891515 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:47.890740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:45747)
0806 22:31:47.891234 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:47.891326 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":262,"threads_started":1}
I20260806 22:31:48.389592 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:48.387688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:41929)
0806 22:31:48.389106 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:48.389242 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1066,"thread_start_us":920,"threads_started":1}
I20260806 22:31:48.859347 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:48.858637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:56897)
0806 22:31:48.859076 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:48.859156 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":233,"threads_started":1}
I20260806 22:31:48.893564 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:48.892852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.65:46059)
0806 22:31:48.893310 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:48.893386 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260806 22:31:49.222117 28118 raft_consensus.cc:482] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:49.222527 28118 raft_consensus.cc:504] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f570ec1b3854ff585185f7e49cc96f4" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40185 } } peers { permanent_uuid: "cf7b854f41d04c8c98b0af720ef28278" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42329 } } peers { permanent_uuid: "6b723a166b064107a91f1763689a105c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 37403 } }
I20260806 22:31:49.223450 28118 leader_election.cc:288] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185), cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329)
W20260806 22:31:49.224594 27440 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f570ec1b3854ff585185f7e49cc96f4 (127.1.8.66:40185): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
W20260806 22:31:49.224798 27444 leader_election.cc:341] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cf7b854f41d04c8c98b0af720ef28278 (127.1.8.67:42329): Not found: Tablet not found: 16eee89100ed472caaf0aae087d817c0
I20260806 22:31:49.224915 27444 leader_election.cc:302] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b723a166b064107a91f1763689a105c; no voters: 0f570ec1b3854ff585185f7e49cc96f4, cf7b854f41d04c8c98b0af720ef28278
I20260806 22:31:49.225220 28118 raft_consensus.cc:2726] T 16eee89100ed472caaf0aae087d817c0 P 6b723a166b064107a91f1763689a105c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:31:49.391683 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:49.390888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.66:58231)
0806 22:31:49.391391 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:49.391491 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":261,"threads_started":1}
I20260806 22:31:49.836192 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 27349
I20260806 22:31:49.859364 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:49.837509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.1.8.65:37403)
0806 22:31:49.838243 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:49.838252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:49.838284 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:31:49.838384 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:49.838393 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:49.838830 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:49.839241 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:49.843440 (+ 4199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:49.844895 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:49.846717 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:49.847510 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:49.847651 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:49.851228 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:49.851245 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:49.851262 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:49.851311 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:49.855256 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:49.856565 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:49.856573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:49.856576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:49.856748 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:49.857426 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:49.857430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:49.857432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:49.857861 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:49.858136 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:49.858624 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 22:31:49.858882 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":457,"threads_started":1}
I20260806 22:31:49.861783 27604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:49.862166 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:49.860938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:42857 (local address 127.1.8.67:49677)
0806 22:31:49.861725 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:49.861842 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.126:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":469,"threads_started":1}
I20260806 22:31:49.957317 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 27349
I20260806 22:31:49.991786 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 27619
I20260806 22:31:50.006821 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:49.992749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47062 (local address 127.1.8.66:40185)
0806 22:31:49.993284 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:49.993288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:49.993304 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:31:49.993360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:49.993363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:49.993422 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:49.993557 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:49.994991 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:49.995801 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:49.997238 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:49.997669 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:49.997816 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:50.000620 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:50.000625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:50.000629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:50.000664 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:50.003482 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:50.004557 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:50.004565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:50.004569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:50.004726 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:50.005193 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:50.005197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:50.005199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:50.005480 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:50.005852 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:50.006177 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:31:50.006419 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":290,"threads_started":1}
I20260806 22:31:50.007712 27822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:50.097311 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 27619
I20260806 22:31:50.122726 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 27833
I20260806 22:31:50.137908 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:50.123790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.1.8.67:42329)
0806 22:31:50.124490 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:50.124494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:50.124511 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:31:50.124570 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:50.124574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:50.124642 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:50.124734 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:50.126278 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:50.127130 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:50.128638 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:50.129089 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:50.129185 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:50.132347 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:50.132353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:50.132357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:50.132392 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:50.135290 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:50.136150 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:50.136162 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:50.136165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:50.136312 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:50.136672 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:50.136675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:50.136677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:50.136951 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:50.137170 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:50.137418 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:31:50.137601 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":408,"threads_started":1}
I20260806 22:31:50.138721 28021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:50.227357 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 27833
2026-08-06T22:31:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65160 ms)
[----------] 30 tests from SecurityITest (319108 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:31:50 dist-test-slave-7vjv krb5kdc[28139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:31:50 dist-test-slave-7vjv krb5kdc[28139](info): set up 2 sockets
Aug 06 22:31:50 dist-test-slave-7vjv krb5kdc[28139](info): commencing operation
krb5kdc: starting...
W20260806 22:31:51.623643 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:31:51 dist-test-slave-7vjv krb5kdc[28139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:31:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:31:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:31:51.929761 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:36005
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:36005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:52.275525 28162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:52.275794 28162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:52.275947 28162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:52.282985 28162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:31:52.283098 28162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:52.283211 28162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:52.283316 28162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:31:52.283406 28162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:31:52.293828 28162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:36005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:36005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:52.295125 28162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:52.296541 28162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:52.309010 28173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:52.309144 28168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:52.309109 28170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:52.309304 28162 server_base.cc:1034] running on GCE node
I20260806 22:31:52.310613 28162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:52.311968 28162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:52.313216 28162 hybrid_clock.cc:648] HybridClock initialized: now 1786055512313177 us; error 68 us; skew 500 ppm
I20260806 22:31:52.313585 28162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:52 dist-test-slave-7vjv krb5kdc[28139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:52.316890 28162 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:31:52.318313 28162 webserver.cc:460] Webserver started at http://127.1.8.126:38363/ using document root <none> and password file <none>
I20260806 22:31:52.319000 28162 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:52.319152 28162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:52.319464 28162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:52.322538 28162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "d786406b3a9643f7b3c6bb52a6e812cb"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "0fa753333a7082483019420872610562"
server_key_iv: "f3d7a9924e6045f06f1a1f221d220d0f"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.323323 28162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "d786406b3a9643f7b3c6bb52a6e812cb"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "0fa753333a7082483019420872610562"
server_key_iv: "f3d7a9924e6045f06f1a1f221d220d0f"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.328697 28162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 22:31:52.332159 28182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:52.333506 28162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:31:52.333654 28162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "d786406b3a9643f7b3c6bb52a6e812cb"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "0fa753333a7082483019420872610562"
server_key_iv: "f3d7a9924e6045f06f1a1f221d220d0f"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.333823 28162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:52.362576 28162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:52.366655 28162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:52.366889 28162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:52.385864 28162 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:36005
I20260806 22:31:52.385901 28268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:36005 every 8 connection(s)
I20260806 22:31:52.387266 28162 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:31:52.391291 28271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:52.391448 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 28162
I20260806 22:31:52.391680 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:31:52.392619 1057 external_mini_cluster.cc:1428] Setting key 258d7919105aa8621a336822584b2f48
I20260806 22:31:52.404053 28271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: Bootstrap starting.
Aug 06 22:31:52 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:52.407526 28271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:52.408888 28271 log.cc:824] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:52.412205 28271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: No bootstrap required, opened a new log
I20260806 22:31:52.416558 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:52.394777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.1.8.126:36005)
0806 22:31:52.395592 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:52.395605 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:52.395648 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:31:52.397963 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:52.397985 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:52.398360 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:52.398753 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:52.400389 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:52.401736 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:52.403159 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:52.403817 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:52.404024 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:52.408828 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:52.408835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:52.408848 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:52.408892 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:52.412460 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:52.413308 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:52.413314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:52.413320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:52.413492 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:52.413917 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:52.413920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:52.413922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:52.414440 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:52.414678 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:52.415124 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 22:31:52.415438 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":361,"threads_started":1}
I20260806 22:31:52.420848 28271 raft_consensus.cc:348] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } }
I20260806 22:31:52.421205 28271 raft_consensus.cc:374] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:52.421375 28271 raft_consensus.cc:729] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d786406b3a9643f7b3c6bb52a6e812cb, State: Initialized, Role: FOLLOWER
I20260806 22:31:52.421932 28271 consensus_queue.cc:260] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } }
I20260806 22:31:52.422174 28271 raft_consensus.cc:388] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:31:52.422336 28271 raft_consensus.cc:482] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:31:52.422499 28271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:52.424960 28271 raft_consensus.cc:504] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } }
I20260806 22:31:52.425530 28271 leader_election.cc:302] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d786406b3a9643f7b3c6bb52a6e812cb; no voters:
I20260806 22:31:52.426216 28271 leader_election.cc:288] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:31:52.426390 28277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:31:52.427088 28277 raft_consensus.cc:686] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [term 1 LEADER]: Becoming Leader. State: Replica: d786406b3a9643f7b3c6bb52a6e812cb, State: Running, Role: LEADER
I20260806 22:31:52.427634 28277 consensus_queue.cc:237] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } }
I20260806 22:31:52.428357 28271 sys_catalog.cc:564] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:31:52.430737 28281 sys_catalog.cc:455] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d786406b3a9643f7b3c6bb52a6e812cb. Latest consensus state: current_term: 1 leader_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } } }
I20260806 22:31:52.431205 28281 sys_catalog.cc:458] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:52.431157 28279 sys_catalog.cc:455] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d786406b3a9643f7b3c6bb52a6e812cb" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 36005 } } }
I20260806 22:31:52.431490 28279 sys_catalog.cc:458] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:52.432854 28294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:31:52.438472 28294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:31:52.447880 28294 catalog_manager.cc:1347] Generated new cluster ID: 0302fdd2b2ad40eb9de6e3c29b7d7fda
I20260806 22:31:52.448004 28294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:31:52.467221 28294 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:31:52.469080 28294 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:31:52.485155 28294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: Generated new TSK 0
I20260806 22:31:52.486318 28294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:31:52.572458 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:52.938954 28311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:52.939368 28311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:52.939543 28311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:52.947026 28311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:52.947129 28311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:52.947254 28311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:31:52.957172 28311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:52.958462 28311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:52.959782 28311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:52.973795 28317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:52.973722 28324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:52.973769 28318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:52.974679 28311 server_base.cc:1034] running on GCE node
I20260806 22:31:52.975454 28311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:52.976512 28311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:52.977756 28311 hybrid_clock.cc:648] HybridClock initialized: now 1786055512977723 us; error 47 us; skew 500 ppm
I20260806 22:31:52.978091 28311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:52 dist-test-slave-7vjv krb5kdc[28139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:52.981381 28311 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:31:52.982872 28311 webserver.cc:460] Webserver started at http://127.1.8.65:43315/ using document root <none> and password file <none>
I20260806 22:31:52.983609 28311 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:52.983765 28311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:52.984144 28311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:52.987353 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "72127fd6068540dd8bc72eff1c071fff"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "ff578a60d78751f1bd5c43446fbfca3c"
server_key_iv: "2bc69d135412d80c1eac0ceae6caf151"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.988332 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "72127fd6068540dd8bc72eff1c071fff"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "ff578a60d78751f1bd5c43446fbfca3c"
server_key_iv: "2bc69d135412d80c1eac0ceae6caf151"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.993556 28311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 22:31:52.997255 28334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:52.998519 28311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:31:52.998746 28311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "72127fd6068540dd8bc72eff1c071fff"
format_stamp: "Formatted at 2026-08-06 22:31:52 on dist-test-slave-7vjv"
server_key: "ff578a60d78751f1bd5c43446fbfca3c"
server_key_iv: "2bc69d135412d80c1eac0ceae6caf151"
server_key_version: "encryptionkey@0"
I20260806 22:31:52.998961 28311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:53.014312 28311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:53.018394 28311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:53.018707 28311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:53.019728 28311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:53.021133 28311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:53.021239 28311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:53.021399 28311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:53.021478 28311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:53.055681 28311 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:38133
I20260806 22:31:53.055732 28516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:38133 every 8 connection(s)
I20260806 22:31:53.057024 28311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:31:53.067095 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 28311
I20260806 22:31:53.067317 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:31:53.068212 1057 external_mini_cluster.cc:1428] Setting key d57da04afdad7bdb9776696e4595e016
Aug 06 22:31:53 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:53.077435 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:53.059590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:39141 (local address 127.1.8.126:36005)
0806 22:31:53.060079 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:53.060084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:53.060977 (+ 893us) server_negotiation.cc:407] Connection header received
0806 22:31:53.062818 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:53.062824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:53.062917 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:53.063045 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:53.064526 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.065256 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.066844 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.067309 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.067463 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:53.071478 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:53.071483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:53.071486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:53.071517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:53.074287 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:53.075040 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:53.075051 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:53.075053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:53.075175 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:53.075647 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:53.075650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:53.075652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:53.075911 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:53.076080 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:53.076837 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 22:31:53.077069 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":272,"threads_started":1}
I20260806 22:31:53.078438 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:53.060001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36005 (local address 127.1.8.65:39141)
0806 22:31:53.060790 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:53.060828 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:53.062543 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:53.063292 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:53.063301 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:53.063781 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:53.064344 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:53.064360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.065429 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.065433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:53.066598 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:53.066610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.067594 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.067601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:53.067904 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:53.068702 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:53.068732 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:53.071259 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:53.074440 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:53.074450 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:53.074467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:53.074900 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:53.075295 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:53.075300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:53.075304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:53.075504 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:53.076050 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:53.076058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:53.076438 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 22:31:53.076722 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 22:31:53.077267 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":273,"threads_started":1}
I20260806 22:31:53.080067 28517 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36005
I20260806 22:31:53.080425 28517 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:53.081316 28517 heartbeater.cc:511] Master 127.1.8.126:36005 requested a full tablet report, sending...
I20260806 22:31:53.083487 28209 ts_manager.cc:194] Registered new tserver with Master: 72127fd6068540dd8bc72eff1c071fff (127.1.8.65:38133)
I20260806 22:31:53.085474 28209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:39141
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:31:53.139421 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:53.490800 28526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:53.491164 28526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:53.491323 28526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:53.498600 28526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:53.498773 28526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:53.498965 28526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:31:53.508952 28526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:53.510264 28526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:53.511576 28526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:53.524966 28540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:53.525009 28545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:53.525043 28541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:53.526122 28526 server_base.cc:1034] running on GCE node
I20260806 22:31:53.526722 28526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:53.527626 28526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:53.528887 28526 hybrid_clock.cc:648] HybridClock initialized: now 1786055513528832 us; error 50 us; skew 500 ppm
I20260806 22:31:53.529227 28526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:53 dist-test-slave-7vjv krb5kdc[28139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055513, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:53.535795 28526 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:31:53.537353 28526 webserver.cc:460] Webserver started at http://127.1.8.66:38511/ using document root <none> and password file <none>
I20260806 22:31:53.538122 28526 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:53.538275 28526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:53.538628 28526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:53.541896 28526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "6e1863219fa140efb7083c40c5e6e870"
format_stamp: "Formatted at 2026-08-06 22:31:53 on dist-test-slave-7vjv"
server_key: "9deaf8e79b393008e90aaf6e18df22aa"
server_key_iv: "e57cee092dedcabdef055a385918044d"
server_key_version: "encryptionkey@0"
I20260806 22:31:53.542743 28526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "6e1863219fa140efb7083c40c5e6e870"
format_stamp: "Formatted at 2026-08-06 22:31:53 on dist-test-slave-7vjv"
server_key: "9deaf8e79b393008e90aaf6e18df22aa"
server_key_iv: "e57cee092dedcabdef055a385918044d"
server_key_version: "encryptionkey@0"
I20260806 22:31:53.548264 28526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 22:31:53.551872 28560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:53.553257 28526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:31:53.553470 28526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "6e1863219fa140efb7083c40c5e6e870"
format_stamp: "Formatted at 2026-08-06 22:31:53 on dist-test-slave-7vjv"
server_key: "9deaf8e79b393008e90aaf6e18df22aa"
server_key_iv: "e57cee092dedcabdef055a385918044d"
server_key_version: "encryptionkey@0"
I20260806 22:31:53.553695 28526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:53.582949 28526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:53.588272 28526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:53.588584 28526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:53.589599 28526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:53.591055 28526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:53.591174 28526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:53.591325 28526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:53.591401 28526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:53.626542 28526 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40591
I20260806 22:31:53.626595 28737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40591 every 8 connection(s)
I20260806 22:31:53.627875 28526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:31:53.633062 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 28526
I20260806 22:31:53.633345 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:31:53.633827 1057 external_mini_cluster.cc:1428] Setting key b7c0d2cdb1131a22c320854432f50880
Aug 06 22:31:53 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055513, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:53.649260 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:53.630636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:51709 (local address 127.1.8.126:36005)
0806 22:31:53.631114 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:53.631119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:53.632004 (+ 885us) server_negotiation.cc:407] Connection header received
0806 22:31:53.634244 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:53.634250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:53.634346 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:53.634470 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:53.636308 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.637026 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.638573 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.639113 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.639248 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:53.643310 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:53.643324 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:53.643328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:53.643360 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:53.646037 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:53.646800 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:53.646806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:53.646811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:53.646947 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:53.647409 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:53.647415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:53.647423 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:53.647685 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:53.647850 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:53.648625 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 22:31:53.648864 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":254,"threads_started":1}
I20260806 22:31:53.650241 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:53.631037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36005 (local address 127.1.8.66:51709)
0806 22:31:53.631807 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:53.631857 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:53.633961 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:53.634718 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:53.634727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:53.635467 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:53.636075 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:53.636151 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.637188 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.637193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:53.638423 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:53.638435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:53.639343 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:53.639350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:53.639587 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:53.640381 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:53.640415 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:53.643064 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:53.646200 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:53.646210 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:53.646226 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:53.646663 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:53.647068 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:53.647073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:53.647077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:53.647280 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:53.647822 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:53.647829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:53.648228 (+ 399us) client_negotiation.cc:769] Sending connection context
0806 22:31:53.648503 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 22:31:53.649046 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":288,"threads_started":1}
I20260806 22:31:53.651799 28739 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36005
I20260806 22:31:53.652171 28739 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:53.653045 28739 heartbeater.cc:511] Master 127.1.8.126:36005 requested a full tablet report, sending...
I20260806 22:31:53.654690 28209 ts_manager.cc:194] Registered new tserver with Master: 6e1863219fa140efb7083c40c5e6e870 (127.1.8.66:40591)
I20260806 22:31:53.655535 28209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:51709
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:31:53.707518 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:31:54.060724 28746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:54.061028 28746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:54.061220 28746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:54.068665 28746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:54.068826 28746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:54.069010 28746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:31:54.079114 28746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:36005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:54.080580 28746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:54.081914 28746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:31:54.089303 28517 heartbeater.cc:503] Master 127.1.8.126:36005 was elected leader, sending a full tablet report...
W20260806 22:31:54.096232 28753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:54.096179 28756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:54.096206 28758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:54.096956 28746 server_base.cc:1034] running on GCE node
I20260806 22:31:54.097623 28746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:54.098620 28746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:54.099848 28746 hybrid_clock.cc:648] HybridClock initialized: now 1786055514099811 us; error 52 us; skew 500 ppm
I20260806 22:31:54.100214 28746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:31:54 dist-test-slave-7vjv krb5kdc[28139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:31:54.103456 28746 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:31:54.104929 28746 webserver.cc:460] Webserver started at http://127.1.8.67:38781/ using document root <none> and password file <none>
I20260806 22:31:54.105623 28746 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:54.105770 28746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:54.106161 28746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:54.109112 28746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "e6bb8fbe508a45cfb10a8bd420b41391"
format_stamp: "Formatted at 2026-08-06 22:31:54 on dist-test-slave-7vjv"
server_key: "f16358b32be02855c613001de8ec9c14"
server_key_iv: "beab58d178870286b7d5a9bd518dc581"
server_key_version: "encryptionkey@0"
I20260806 22:31:54.109910 28746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "e6bb8fbe508a45cfb10a8bd420b41391"
format_stamp: "Formatted at 2026-08-06 22:31:54 on dist-test-slave-7vjv"
server_key: "f16358b32be02855c613001de8ec9c14"
server_key_iv: "beab58d178870286b7d5a9bd518dc581"
server_key_version: "encryptionkey@0"
I20260806 22:31:54.114995 28746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:31:54.118410 28767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:54.119709 28746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:31:54.119889 28746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "e6bb8fbe508a45cfb10a8bd420b41391"
format_stamp: "Formatted at 2026-08-06 22:31:54 on dist-test-slave-7vjv"
server_key: "f16358b32be02855c613001de8ec9c14"
server_key_iv: "beab58d178870286b7d5a9bd518dc581"
server_key_version: "encryptionkey@0"
I20260806 22:31:54.120148 28746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:54.140990 28746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:54.145056 28746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:54.145365 28746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:54.146381 28746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:54.147727 28746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:54.147840 28746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:54.147993 28746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:54.148075 28746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:54.182696 28746 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:42611
I20260806 22:31:54.182755 28942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:42611 every 8 connection(s)
I20260806 22:31:54.184069 28746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:31:54.191506 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 28746
I20260806 22:31:54.191740 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:31:54.192394 1057 external_mini_cluster.cc:1428] Setting key db49729901ca027fec392a37c2c6b63e
Aug 06 22:31:54 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:31:54.204772 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.186783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:42955 (local address 127.1.8.126:36005)
0806 22:31:54.187255 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.187260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.188125 (+ 865us) server_negotiation.cc:407] Connection header received
0806 22:31:54.190194 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.190200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.190294 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.190436 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:54.191995 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.192930 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.194470 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.194925 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.195065 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.198594 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:54.198598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:54.198601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:54.198632 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:54.201320 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.202030 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.202037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.202040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.202166 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.202644 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.202650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.202653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.202943 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:54.203312 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.204150 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.204389 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260806 22:31:54.205763 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.187237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:36005 (local address 127.1.8.67:42955)
0806 22:31:54.187937 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.187987 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.189905 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.190667 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.190674 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.191191 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:54.191796 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.191830 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.193080 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.193085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.194311 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.194322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.195171 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.195178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.195405 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.196163 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:31:54.196191 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:54.198379 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:54.201470 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:54.201478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:54.201494 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:54.201912 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:54.202297 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:31:54.202302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:31:54.202306 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:31:54.202533 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:31:54.203276 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:54.203284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:54.203713 (+ 429us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.203987 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.204561 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":282,"threads_started":1}
I20260806 22:31:54.207296 28944 heartbeater.cc:347] Connected to a master server at 127.1.8.126:36005
I20260806 22:31:54.207679 28944 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:54.208523 28944 heartbeater.cc:511] Master 127.1.8.126:36005 requested a full tablet report, sending...
I20260806 22:31:54.209967 28206 ts_manager.cc:194] Registered new tserver with Master: e6bb8fbe508a45cfb10a8bd420b41391 (127.1.8.67:42611)
I20260806 22:31:54.210806 28206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:42955
I20260806 22:31:54.220611 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:31:54.238421 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.225283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.1.8.126:36005)
0806 22:31:54.225423 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.225427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.225910 (+ 483us) server_negotiation.cc:407] Connection header received
0806 22:31:54.226153 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.226157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.226221 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.226322 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:54.227782 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.228591 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.229972 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.230372 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.230527 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.233179 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:54.233184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:54.233187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:54.233220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:54.235623 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.236370 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.236376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.236380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.236525 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.236909 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.236914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.236917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.237248 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:54.237533 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.237859 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.238105 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:31:54.243211 28206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:31:54.250968 28206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:31:54.285066 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.263485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42611 (local address 127.0.0.1:54174)
0806 22:31:54.275074 (+ 11589us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.275096 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.275286 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.277569 (+ 2283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.277577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.277612 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:54.278402 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.278420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.280494 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.280500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.282604 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.282615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.283891 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.283897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.284231 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.284271 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.284413 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.284612 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11420,"mutex_wait_us":99,"thread_start_us":10293,"threads_started":1}
I20260806 22:31:54.285303 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.264144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.1.8.67:42611)
0806 22:31:54.273551 (+ 9407us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.273564 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.275660 (+ 2096us) server_negotiation.cc:407] Connection header received
0806 22:31:54.275801 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.275813 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.277147 (+ 1334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.277323 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:54.278666 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.280368 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.282766 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.283688 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.283831 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.284101 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.284462 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.284644 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9151,"thread_start_us":8984,"threads_started":1}
I20260806 22:31:54.285933 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.263292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40591 (local address 127.0.0.1:50624)
0806 22:31:54.274334 (+ 11042us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.274410 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.275498 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.277294 (+ 1796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.277307 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.277354 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:54.278191 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.278212 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.279963 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.279969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.282663 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.282678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.284873 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.284879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.285058 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.285093 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.285306 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.285493 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10654,"mutex_wait_us":59,"thread_start_us":11410,"threads_started":1}
I20260806 22:31:54.286810 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.263512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.1.8.66:40591)
0806 22:31:54.276608 (+ 13096us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.276619 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.276652 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:31:54.276758 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.276766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.276992 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.277238 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:54.278434 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.279809 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.282873 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.284411 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.284542 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.284725 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.286255 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.286462 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12920,"thread_start_us":12738,"threads_started":1}
I20260806 22:31:54.287551 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.275616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.1.8.65:38133)
0806 22:31:54.276992 (+ 1376us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.277001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.278490 (+ 1489us) server_negotiation.cc:407] Connection header received
0806 22:31:54.278785 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.278794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.279029 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.279215 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:54.280171 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.281727 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.284375 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.285847 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.285955 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.286193 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.287057 (+ 864us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.287228 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":1009,"threads_started":1}
I20260806 22:31:54.288259 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.275160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38133 (local address 127.0.0.1:44170)
0806 22:31:54.278371 (+ 3211us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.278395 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.278597 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.279509 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.279514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.279537 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:54.279949 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.279960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.281897 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.281904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.284187 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.284202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.286652 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.286657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.286831 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.286871 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.287822 (+ 951us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.287930 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3035,"mutex_wait_us":51,"thread_start_us":2806,"threads_started":1}
I20260806 22:31:54.290369 28839 tablet_service.cc:1467] Processing CreateTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a (DEFAULT_TABLE table=test-table [id=026fd4b4a4454561ae135f28d946917f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:54.290655 28637 tablet_service.cc:1467] Processing CreateTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a (DEFAULT_TABLE table=test-table [id=026fd4b4a4454561ae135f28d946917f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:54.292021 28839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f8f075907e45d0b020b9fd5ec92b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:54.292038 28637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f8f075907e45d0b020b9fd5ec92b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:54.291628 28415 tablet_service.cc:1467] Processing CreateTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a (DEFAULT_TABLE table=test-table [id=026fd4b4a4454561ae135f28d946917f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:54.293031 28415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f8f075907e45d0b020b9fd5ec92b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:54.306020 28969 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Bootstrap starting.
I20260806 22:31:54.308902 28973 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Bootstrap starting.
I20260806 22:31:54.310451 28969 tablet_bootstrap.cc:654] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:54.311861 28969 log.cc:824] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:54.312739 28973 tablet_bootstrap.cc:654] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:54.314297 28973 log.cc:824] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:54.315271 28971 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Bootstrap starting.
I20260806 22:31:54.318231 28971 tablet_bootstrap.cc:654] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:54.319558 28971 log.cc:824] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:54.321583 28973 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: No bootstrap required, opened a new log
I20260806 22:31:54.321811 28969 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: No bootstrap required, opened a new log
I20260806 22:31:54.321895 28973 ts_tablet_manager.cc:1397] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260806 22:31:54.322167 28969 ts_tablet_manager.cc:1397] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260806 22:31:54.322939 28971 tablet_bootstrap.cc:492] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: No bootstrap required, opened a new log
I20260806 22:31:54.323194 28971 ts_tablet_manager.cc:1397] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 22:31:54.329952 28973 raft_consensus.cc:348] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.330374 28973 raft_consensus.cc:374] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:54.330470 28973 raft_consensus.cc:729] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bb8fbe508a45cfb10a8bd420b41391, State: Initialized, Role: FOLLOWER
I20260806 22:31:54.331257 28973 consensus_queue.cc:260] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.333436 28969 raft_consensus.cc:348] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.333889 28969 raft_consensus.cc:374] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:54.334010 28969 raft_consensus.cc:729] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72127fd6068540dd8bc72eff1c071fff, State: Initialized, Role: FOLLOWER
I20260806 22:31:54.334132 28971 raft_consensus.cc:348] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.334548 28971 raft_consensus.cc:374] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:54.334650 28971 raft_consensus.cc:729] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1863219fa140efb7083c40c5e6e870, State: Initialized, Role: FOLLOWER
I20260806 22:31:54.334671 28969 consensus_queue.cc:260] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.341059 28971 consensus_queue.cc:260] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.349117 28971 ts_tablet_manager.cc:1428] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Time spent starting tablet: real 0.026s user 0.015s sys 0.001s
I20260806 22:31:54.350205 28739 heartbeater.cc:503] Master 127.1.8.126:36005 was elected leader, sending a full tablet report...
I20260806 22:31:54.350737 28944 heartbeater.cc:503] Master 127.1.8.126:36005 was elected leader, sending a full tablet report...
I20260806 22:31:54.351384 28973 ts_tablet_manager.cc:1428] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Time spent starting tablet: real 0.029s user 0.014s sys 0.016s
I20260806 22:31:54.361362 28969 ts_tablet_manager.cc:1428] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Time spent starting tablet: real 0.039s user 0.024s sys 0.014s
I20260806 22:31:54.364954 28984 raft_consensus.cc:482] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:54.365176 28984 raft_consensus.cc:504] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.366845 28984 leader_election.cc:288] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1863219fa140efb7083c40c5e6e870 (127.1.8.66:40591), e6bb8fbe508a45cfb10a8bd420b41391 (127.1.8.67:42611)
I20260806 22:31:54.374106 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.367422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:47843 (local address 127.1.8.67:42611)
0806 22:31:54.367710 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.367718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.368233 (+ 515us) server_negotiation.cc:407] Connection header received
0806 22:31:54.368525 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.368530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.368609 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.368700 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:54.369380 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.370318 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.372325 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.373113 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.373282 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.373400 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.373693 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.373824 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:31:54.374260 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.367157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44563 (local address 127.1.8.66:40591)
0806 22:31:54.367396 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.367405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.368519 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 22:31:54.368739 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.368744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.368822 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.368922 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:54.369613 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.370297 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.372320 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.373223 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.373382 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.373495 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.373775 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.373905 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:31:54.374295 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.367140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42611 (local address 127.1.8.65:47843)
0806 22:31:54.368117 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.368141 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.368357 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.368751 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.368755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.368777 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:54.369228 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.369240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.370481 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.370485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.372180 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.372193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.373348 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.373355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.373519 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.373546 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.373649 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.373773 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":33,"thread_start_us":1006,"threads_started":1}
I20260806 22:31:54.374603 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.367236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40591 (local address 127.1.8.65:44563)
0806 22:31:54.368421 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:54.368438 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:54.368598 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:54.368966 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:54.368971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:54.369005 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:54.369447 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.369463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.370441 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.370447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.372157 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:54.372189 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.373455 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.373461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:54.373614 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.373636 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:31:54.373737 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:31:54.373854 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":111,"thread_start_us":481,"threads_started":1}
I20260806 22:31:54.375415 28872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f8f075907e45d0b020b9fd5ec92b1a" candidate_uuid: "72127fd6068540dd8bc72eff1c071fff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" is_pre_election: true
I20260806 22:31:54.375586 28668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f8f075907e45d0b020b9fd5ec92b1a" candidate_uuid: "72127fd6068540dd8bc72eff1c071fff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1863219fa140efb7083c40c5e6e870" is_pre_election: true
I20260806 22:31:54.376117 28668 raft_consensus.cc:2454] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72127fd6068540dd8bc72eff1c071fff in term 0.
I20260806 22:31:54.376058 28872 raft_consensus.cc:2454] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72127fd6068540dd8bc72eff1c071fff in term 0.
I20260806 22:31:54.376930 28339 leader_election.cc:302] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e1863219fa140efb7083c40c5e6e870, 72127fd6068540dd8bc72eff1c071fff; no voters:
I20260806 22:31:54.377360 28984 raft_consensus.cc:2781] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:31:54.377512 28984 raft_consensus.cc:482] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:31:54.377624 28984 raft_consensus.cc:3037] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:54.379850 28984 raft_consensus.cc:504] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
W20260806 22:31:54.380388 28740 tablet.cc:2369] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:31:54.380774 28984 leader_election.cc:288] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [CANDIDATE]: Term 1 election: Requested vote from peers 6e1863219fa140efb7083c40c5e6e870 (127.1.8.66:40591), e6bb8fbe508a45cfb10a8bd420b41391 (127.1.8.67:42611)
I20260806 22:31:54.380964 28668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f8f075907e45d0b020b9fd5ec92b1a" candidate_uuid: "72127fd6068540dd8bc72eff1c071fff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1863219fa140efb7083c40c5e6e870"
I20260806 22:31:54.381069 28872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f8f075907e45d0b020b9fd5ec92b1a" candidate_uuid: "72127fd6068540dd8bc72eff1c071fff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bb8fbe508a45cfb10a8bd420b41391"
I20260806 22:31:54.381174 28668 raft_consensus.cc:3037] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:54.381254 28872 raft_consensus.cc:3037] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:54.384485 28872 raft_consensus.cc:2454] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72127fd6068540dd8bc72eff1c071fff in term 1.
I20260806 22:31:54.384485 28668 raft_consensus.cc:2454] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72127fd6068540dd8bc72eff1c071fff in term 1.
I20260806 22:31:54.384941 28339 leader_election.cc:302] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6e1863219fa140efb7083c40c5e6e870, 72127fd6068540dd8bc72eff1c071fff, e6bb8fbe508a45cfb10a8bd420b41391; no voters:
I20260806 22:31:54.385298 28984 raft_consensus.cc:2781] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:31:54.385804 28984 raft_consensus.cc:686] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 1 LEADER]: Becoming Leader. State: Replica: 72127fd6068540dd8bc72eff1c071fff, State: Running, Role: LEADER
I20260806 22:31:54.386324 28984 consensus_queue.cc:237] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } }
I20260806 22:31:54.391388 28209 catalog_manager.cc:5495] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff reported cstate change: term changed from 0 to 1, leader changed from <none> to 72127fd6068540dd8bc72eff1c071fff (127.1.8.65). New cstate: current_term: 1 leader_uuid: "72127fd6068540dd8bc72eff1c071fff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72127fd6068540dd8bc72eff1c071fff" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38133 } health_report { overall_health: HEALTHY } } }
I20260806 22:31:54.425236 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.419319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.1.8.65:38133)
0806 22:31:54.419538 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.419546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.419664 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:31:54.419924 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.419928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.420032 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.420234 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:31:54.420632 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.421379 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.422973 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.423367 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.423512 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.424056 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:31:54.424648 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:31:54.424811 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.424872 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.424951 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:31:54.433826 28668 raft_consensus.cc:1261] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 1 FOLLOWER]: Refusing update from remote peer 72127fd6068540dd8bc72eff1c071fff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:54.433818 28872 raft_consensus.cc:1261] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 1 FOLLOWER]: Refusing update from remote peer 72127fd6068540dd8bc72eff1c071fff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:54.434953 28993 consensus_queue.cc:1035] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6bb8fbe508a45cfb10a8bd420b41391" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:31:54.435325 28984 consensus_queue.cc:1035] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1863219fa140efb7083c40c5e6e870" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:31:54.435747 28946 tablet.cc:2369] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:31:54.447014 28996 mvcc.cc:205] Tried to move back new op lower bound from 7315683387109515264 to 7315683386930155520. Current Snapshot: MvccSnapshot[applied={T|T < 7315683387109515264}]
I20260806 22:31:54.448500 28999 mvcc.cc:205] Tried to move back new op lower bound from 7315683387109515264 to 7315683386930155520. Current Snapshot: MvccSnapshot[applied={T|T < 7315683387109515264}]
I20260806 22:31:54.456281 28998 mvcc.cc:205] Tried to move back new op lower bound from 7315683387109515264 to 7315683386930155520. Current Snapshot: MvccSnapshot[applied={T|T < 7315683387109515264}]
I20260806 22:31:54.461580 28206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:54.461854 28206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:54.465255 28206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d786406b3a9643f7b3c6bb52a6e812cb: Sending DeleteTablet for 3 replicas of tablet a2f8f075907e45d0b020b9fd5ec92b1a
I20260806 22:31:54.466427 28839 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:54 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:54174
I20260806 22:31:54.466480 28637 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:54 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 22:31:54.466563 28415 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f8f075907e45d0b020b9fd5ec92b1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:54 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:44170
I20260806 22:31:54.467392 29014 tablet_replica.cc:330] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: stopping tablet replica
I20260806 22:31:54.467916 29014 raft_consensus.cc:2229] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:54.468662 29014 raft_consensus.cc:2258] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:54.469000 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 28311
I20260806 22:31:54.475550 29014 ts_tablet_manager.cc:1905] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:54.488382 29017 tablet_replica.cc:330] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: stopping tablet replica
I20260806 22:31:54.489053 29017 raft_consensus.cc:2229] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:54.490639 29017 raft_consensus.cc:2258] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:54.490813 29020 tablet_replica.cc:330] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: stopping tablet replica
I20260806 22:31:54.491490 29020 raft_consensus.cc:2229] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:31:54.492242 29020 raft_consensus.cc:2258] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:54.494071 29014 ts_tablet_manager.cc:1918] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:54.494270 29014 log.cc:1196] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/a2f8f075907e45d0b020b9fd5ec92b1a
I20260806 22:31:54.494699 29020 ts_tablet_manager.cc:1905] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:54.494858 29014 ts_tablet_manager.cc:1939] T a2f8f075907e45d0b020b9fd5ec92b1a P e6bb8fbe508a45cfb10a8bd420b41391: Deleting consensus metadata
I20260806 22:31:54.496914 28185 catalog_manager.cc:4841] TS e6bb8fbe508a45cfb10a8bd420b41391 (127.1.8.67:42611): tablet a2f8f075907e45d0b020b9fd5ec92b1a (table test-table [id=026fd4b4a4454561ae135f28d946917f]) successfully deleted
I20260806 22:31:54.498378 29017 ts_tablet_manager.cc:1905] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:31:54 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:31:54.502275 29020 ts_tablet_manager.cc:1918] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:54.502470 29020 log.cc:1196] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/a2f8f075907e45d0b020b9fd5ec92b1a
I20260806 22:31:54.503113 29020 ts_tablet_manager.cc:1939] T a2f8f075907e45d0b020b9fd5ec92b1a P 72127fd6068540dd8bc72eff1c071fff: Deleting consensus metadata
I20260806 22:31:54.504978 28187 catalog_manager.cc:4841] TS 72127fd6068540dd8bc72eff1c071fff (127.1.8.65:38133): tablet a2f8f075907e45d0b020b9fd5ec92b1a (table test-table [id=026fd4b4a4454561ae135f28d946917f]) successfully deleted
I20260806 22:31:54.505421 29017 ts_tablet_manager.cc:1918] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:54.505610 29017 log.cc:1196] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/a2f8f075907e45d0b020b9fd5ec92b1a
I20260806 22:31:54.506065 29017 ts_tablet_manager.cc:1939] T a2f8f075907e45d0b020b9fd5ec92b1a P 6e1863219fa140efb7083c40c5e6e870: Deleting consensus metadata
I20260806 22:31:54.507572 28187 catalog_manager.cc:4841] TS 6e1863219fa140efb7083c40c5e6e870 (127.1.8.66:40591): tablet a2f8f075907e45d0b020b9fd5ec92b1a (table test-table [id=026fd4b4a4454561ae135f28d946917f]) successfully deleted
I20260806 22:31:54.510130 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.470238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.1.8.65:38133)
0806 22:31:54.470435 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.470441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.488760 (+ 18319us) server_negotiation.cc:407] Connection header received
0806 22:31:54.488984 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.488990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.489084 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.489198 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:54.491278 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.492681 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.494790 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.495566 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.495693 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.502458 (+ 6765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:54.502475 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:54.502493 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:54.502587 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:54.507066 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.507928 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.507935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.507937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.508139 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.508689 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.508692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.508694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.509049 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:54.509288 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.509652 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.509826 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:31:54.510950 28506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:54.609691 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 28311
I20260806 22:31:54.637266 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 28526
Aug 06 22:31:54 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:31:54.657668 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.638017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.1.8.66:40591)
0806 22:31:54.638193 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.638198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.638311 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:31:54.638434 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.638437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.638519 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.638604 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:54.640287 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.641082 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.642940 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.643321 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.643412 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.652334 (+ 8922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:54.652346 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:54.652359 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:54.652400 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:54.655088 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.655755 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.655762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.655764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.655881 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.656240 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.656243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.656245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.656647 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:54.656873 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.657239 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.657397 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:31:54.658425 28732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:54.748337 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 28526
I20260806 22:31:54.773008 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 28746
Aug 06 22:31:54 dist-test-slave-7vjv krb5kdc[28139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:31:54.788146 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:54.773824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.1.8.67:42611)
0806 22:31:54.774027 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:54.774033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:54.774053 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:31:54.774249 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:54.774255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:54.774376 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:54.774487 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:54.775723 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.776536 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.777807 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:54.778211 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:54.778343 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:54.782809 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:54.782823 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:54.782840 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:31:54.782871 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:54.785422 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.786099 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.786106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.786108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.786236 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:31:54.786619 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:31:54.786625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:31:54.786628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:31:54.786963 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:54.787278 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:54.787566 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:31:54.787789 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:31:54.788806 28936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:54.881974 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 28746
I20260806 22:31:54.906361 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 28162
I20260806 22:31:54.907460 28258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:55.018433 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 28162
2026-08-06T22:31:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4799 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:31:55Z Disabled control of system clock
I20260806 22:31:55.124696 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45483
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:45483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:31:55.473313 29032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:55.473606 29032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:55.473686 29032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:55.480607 29032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:31:55.480711 29032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:55.480749 29032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:55.480808 29032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:31:55.480850 29032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:31:55.489585 29032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:45483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:45483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:55.490768 29032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:55.492528 29032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:55.504590 29044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:55.504648 29042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:55.504760 29040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:55.505764 29032 server_base.cc:1034] running on GCE node
I20260806 22:31:55.506524 29032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:55.507870 29032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:55.509102 29032 hybrid_clock.cc:648] HybridClock initialized: now 1786055515509062 us; error 25 us; skew 500 ppm
I20260806 22:31:55.509471 29032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:31:55.511641 29032 webserver.cc:460] Webserver started at http://127.1.8.126:36859/ using document root <none> and password file <none>
I20260806 22:31:55.512377 29032 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:55.512508 29032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:55.512832 29032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:55.515659 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "49ad20669dbf49f4afe1f1a252e27056"
format_stamp: "Formatted at 2026-08-06 22:31:55 on dist-test-slave-7vjv"
server_key: "d98893408c40f91c2ff9f19dd98536ad"
server_key_iv: "c867dc056fe8f1bf35fb3d2b9c818f3f"
server_key_version: "encryptionkey@0"
I20260806 22:31:55.516511 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "49ad20669dbf49f4afe1f1a252e27056"
format_stamp: "Formatted at 2026-08-06 22:31:55 on dist-test-slave-7vjv"
server_key: "d98893408c40f91c2ff9f19dd98536ad"
server_key_iv: "c867dc056fe8f1bf35fb3d2b9c818f3f"
server_key_version: "encryptionkey@0"
I20260806 22:31:55.521379 29032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:31:55.524775 29056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:55.525982 29032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 22:31:55.526196 29032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "49ad20669dbf49f4afe1f1a252e27056"
format_stamp: "Formatted at 2026-08-06 22:31:55 on dist-test-slave-7vjv"
server_key: "d98893408c40f91c2ff9f19dd98536ad"
server_key_iv: "c867dc056fe8f1bf35fb3d2b9c818f3f"
server_key_version: "encryptionkey@0"
I20260806 22:31:55.526409 29032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:55.547016 29032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:55.547799 29032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:55.548147 29032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:55.566828 29032 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:45483
I20260806 22:31:55.566835 29150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:45483 every 8 connection(s)
I20260806 22:31:55.568318 29032 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:31:55.572149 29152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:55.574420 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 29032
I20260806 22:31:55.574620 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:31:55.575196 1057 external_mini_cluster.cc:1428] Setting key f3a2b96aa66ad33605d3dbb7f3af1c87
I20260806 22:31:55.584654 29152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: Bootstrap starting.
I20260806 22:31:55.588408 29152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:55.589403 29157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:55.577155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.1.8.126:45483)
0806 22:31:55.578274 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:55.578287 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:55.578329 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:31:55.579950 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:55.579971 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:55.580479 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:55.580806 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:55.581381 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:55.582671 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:55.584615 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:55.585330 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:55.585458 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:55.586757 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:55.586764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:55.586776 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:55.587280 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:55.587367 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:55.587387 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:55.587475 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:55.587728 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:31:55.588068 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":489,"threads_started":1}
I20260806 22:31:55.589519 29152 log.cc:824] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:55.592651 29152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: No bootstrap required, opened a new log
I20260806 22:31:55.599885 29152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } }
I20260806 22:31:55.600312 29152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:55.600423 29152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ad20669dbf49f4afe1f1a252e27056, State: Initialized, Role: FOLLOWER
I20260806 22:31:55.600963 29152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } }
I20260806 22:31:55.601213 29152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:31:55.601352 29152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:31:55.601528 29152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:55.603775 29152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } }
I20260806 22:31:55.604365 29152 leader_election.cc:302] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ad20669dbf49f4afe1f1a252e27056; no voters:
I20260806 22:31:55.604933 29152 leader_election.cc:288] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:31:55.605104 29160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:31:55.605818 29160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [term 1 LEADER]: Becoming Leader. State: Replica: 49ad20669dbf49f4afe1f1a252e27056, State: Running, Role: LEADER
I20260806 22:31:55.606359 29160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } }
I20260806 22:31:55.607095 29152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:31:55.609145 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ad20669dbf49f4afe1f1a252e27056. Latest consensus state: current_term: 1 leader_uuid: "49ad20669dbf49f4afe1f1a252e27056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } } }
I20260806 22:31:55.609625 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:55.610025 29161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49ad20669dbf49f4afe1f1a252e27056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ad20669dbf49f4afe1f1a252e27056" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 45483 } } }
I20260806 22:31:55.610344 29161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056 [sys.catalog]: This master's current role is: LEADER
I20260806 22:31:55.611840 29172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:31:55.617518 29172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:31:55.626264 29172 catalog_manager.cc:1347] Generated new cluster ID: ffd51a76086248c9857108eec15679f2
I20260806 22:31:55.626370 29172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:31:55.636494 29172 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:31:55.637651 29172 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:31:55.643757 29172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: Generated new TSK 0
I20260806 22:31:55.644534 29172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:31:55.649019 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--builtin_ntp_servers=127.1.8.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:31:55.994944 29188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:55.995230 29188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:55.995316 29188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:56.002626 29188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:56.002724 29188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:56.002880 29188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:31:56.012223 29188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:56.013590 29188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:56.015404 29188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:56.028560 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.028612 29195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.028648 29194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:56.029805 29188 server_base.cc:1034] running on GCE node
I20260806 22:31:56.030426 29188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:56.031487 29188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:56.032711 29188 hybrid_clock.cc:648] HybridClock initialized: now 1786055516032670 us; error 32 us; skew 500 ppm
I20260806 22:31:56.033046 29188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:31:56.035272 29188 webserver.cc:460] Webserver started at http://127.1.8.65:35351/ using document root <none> and password file <none>
I20260806 22:31:56.035959 29188 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:56.036114 29188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:56.036463 29188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:56.039311 29188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "bbaec8e624ed433995bb7c995ef8f167"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "ae9dd3b32a3f4ddc24b506da0e6316f2"
server_key_iv: "774137178fcf74f0b0b87eaf5c788a96"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.040189 29188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "bbaec8e624ed433995bb7c995ef8f167"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "ae9dd3b32a3f4ddc24b506da0e6316f2"
server_key_iv: "774137178fcf74f0b0b87eaf5c788a96"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.045819 29188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:31:56.049125 29204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.050412 29188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:31:56.050546 29188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "bbaec8e624ed433995bb7c995ef8f167"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "ae9dd3b32a3f4ddc24b506da0e6316f2"
server_key_iv: "774137178fcf74f0b0b87eaf5c788a96"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.050717 29188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:56.064414 29188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:56.065174 29188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:56.065397 29188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:56.066268 29188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:56.067698 29188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:56.067785 29188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.067867 29188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:56.067922 29188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.100462 29188 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:44797
I20260806 22:31:56.100517 29401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:44797 every 8 connection(s)
I20260806 22:31:56.101759 29188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:31:56.103395 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 29188
I20260806 22:31:56.103605 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:31:56.104182 1057 external_mini_cluster.cc:1428] Setting key 84b7f999001567f60e9f2cf024493cd8
I20260806 22:31:56.107527 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--builtin_ntp_servers=127.1.8.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:31:56.116691 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:56.104570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44995 (local address 127.1.8.126:45483)
0806 22:31:56.105072 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:56.105078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:56.109073 (+ 3995us) server_negotiation.cc:407] Connection header received
0806 22:31:56.109158 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:56.109166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:56.109267 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:56.109394 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:56.110422 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.111692 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.113535 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.114283 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.114430 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:56.115724 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:56.115732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:56.115740 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:56.115765 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:56.115829 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:56.115848 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:56.115946 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:56.116172 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:31:56.116279 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
I20260806 22:31:56.117481 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:56.105163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45483 (local address 127.1.8.65:44995)
0806 22:31:56.106209 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:56.106257 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:56.108721 (+ 2464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:56.109615 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:56.109623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:56.109707 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:56.110245 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:56.110262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.111857 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.111862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:56.113388 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:56.113400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.114486 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.114491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:56.114693 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:56.115416 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:31:56.115419 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:56.115457 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:31:56.115535 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:56.115947 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:56.115955 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:56.115961 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:31:56.116144 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 22:31:56.116434 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":330,"threads_started":1}
I20260806 22:31:56.119072 29402 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45483
I20260806 22:31:56.119413 29402 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:56.120401 29402 heartbeater.cc:511] Master 127.1.8.126:45483 requested a full tablet report, sending...
I20260806 22:31:56.122601 29089 ts_manager.cc:194] Registered new tserver with Master: bbaec8e624ed433995bb7c995ef8f167 (127.1.8.65:44797)
I20260806 22:31:56.124048 29089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.65:44995
W20260806 22:31:56.459826 29406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:56.460173 29406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:56.460299 29406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:56.467553 29406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:56.467666 29406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:56.467782 29406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:31:56.477226 29406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:56.478590 29406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:56.480491 29406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:56.494283 29421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.494251 29416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.494318 29415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:56.495260 29406 server_base.cc:1034] running on GCE node
I20260806 22:31:56.496017 29406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:56.497207 29406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:56.498459 29406 hybrid_clock.cc:648] HybridClock initialized: now 1786055516498405 us; error 58 us; skew 500 ppm
I20260806 22:31:56.498790 29406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:31:56.501246 29406 webserver.cc:460] Webserver started at http://127.1.8.66:39943/ using document root <none> and password file <none>
I20260806 22:31:56.502053 29406 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:56.502192 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:56.502534 29406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:56.505558 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "2df494173886431a89823da2e437d832"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "622b840107fed913410324f6602d3688"
server_key_iv: "1466f92d36330b4a42f750f6a2720049"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.506388 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "2df494173886431a89823da2e437d832"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "622b840107fed913410324f6602d3688"
server_key_iv: "1466f92d36330b4a42f750f6a2720049"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.512017 29406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:31:56.516044 29432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.517555 29406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:31:56.517755 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "2df494173886431a89823da2e437d832"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "622b840107fed913410324f6602d3688"
server_key_iv: "1466f92d36330b4a42f750f6a2720049"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.518023 29406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:56.536336 29406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:56.537231 29406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:56.537539 29406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:56.538589 29406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:56.540298 29406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:56.540432 29406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.540566 29406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:56.540658 29406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.575690 29406 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:38779
I20260806 22:31:56.575723 29611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:38779 every 8 connection(s)
I20260806 22:31:56.577327 29406 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:31:56.583465 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 29406
I20260806 22:31:56.583668 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:31:56.584265 1057 external_mini_cluster.cc:1428] Setting key 4801ae2b2dd4f3396b290edc4a071ca2
I20260806 22:31:56.587232 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--builtin_ntp_servers=127.1.8.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:31:56.591518 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:56.580017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:36391 (local address 127.1.8.126:45483)
0806 22:31:56.580269 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:56.580276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:56.581866 (+ 1590us) server_negotiation.cc:407] Connection header received
0806 22:31:56.583915 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:56.583921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:56.584004 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:56.584146 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:56.585502 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.586381 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.588328 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.589100 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.589251 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:56.590636 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:56.590645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:56.590652 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:56.590677 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:56.590720 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:56.590738 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:56.590835 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:56.591038 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:31:56.591142 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 22:31:56.592495 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:56.580612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45483 (local address 127.1.8.66:36391)
0806 22:31:56.581683 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:56.581732 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:56.583607 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:56.584420 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:56.584431 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:56.584544 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:56.585324 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:56.585350 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.586552 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.586558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:56.588167 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:56.588182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:56.589336 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:56.589342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:56.589564 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:56.590320 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:31:56.590322 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:56.590361 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:31:56.590433 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:56.590839 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:56.590847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:56.590853 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:31:56.590992 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:31:56.591287 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":360,"threads_started":1}
I20260806 22:31:56.593971 29612 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45483
I20260806 22:31:56.594323 29612 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:56.595310 29612 heartbeater.cc:511] Master 127.1.8.126:45483 requested a full tablet report, sending...
I20260806 22:31:56.597002 29089 ts_manager.cc:194] Registered new tserver with Master: 2df494173886431a89823da2e437d832 (127.1.8.66:38779)
I20260806 22:31:56.597931 29089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.66:36391
W20260806 22:31:56.941973 29616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:31:56.942242 29616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:31:56.942389 29616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:31:56.949708 29616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:31:56.949808 29616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:31:56.949967 29616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:31:56.959127 29616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:45483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:31:56.960533 29616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:31:56.962337 29616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:31:56.975843 29622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.975762 29623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:31:56.975853 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:31:56.976636 29616 server_base.cc:1034] running on GCE node
I20260806 22:31:56.977301 29616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:31:56.978183 29616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:31:56.979380 29616 hybrid_clock.cc:648] HybridClock initialized: now 1786055516979316 us; error 65 us; skew 500 ppm
I20260806 22:31:56.979719 29616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:31:56.982021 29616 webserver.cc:460] Webserver started at http://127.1.8.67:43899/ using document root <none> and password file <none>
I20260806 22:31:56.982838 29616 fs_manager.cc:359] Metadata directory not provided
I20260806 22:31:56.982928 29616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:31:56.983292 29616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:31:56.986269 29616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "ec089260b5194b138caabd72aecd17e9"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "f38b307610e797d65e1fb6c69408256d"
server_key_iv: "39654a513b1d8b1259eff42cbe416113"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.987073 29616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "ec089260b5194b138caabd72aecd17e9"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "f38b307610e797d65e1fb6c69408256d"
server_key_iv: "39654a513b1d8b1259eff42cbe416113"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.992225 29616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 22:31:56.995702 29635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:56.997066 29616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 22:31:56.997246 29616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "ec089260b5194b138caabd72aecd17e9"
format_stamp: "Formatted at 2026-08-06 22:31:56 on dist-test-slave-7vjv"
server_key: "f38b307610e797d65e1fb6c69408256d"
server_key_iv: "39654a513b1d8b1259eff42cbe416113"
server_key_version: "encryptionkey@0"
I20260806 22:31:56.997468 29616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:31:57.018669 29616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:31:57.019562 29616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:31:57.019877 29616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:31:57.020953 29616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:31:57.022653 29616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:31:57.022845 29616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:57.022989 29616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:31:57.023090 29616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:31:57.057397 29616 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:44503
I20260806 22:31:57.057499 29811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:44503 every 8 connection(s)
I20260806 22:31:57.058740 29616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:31:57.064224 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 29616
I20260806 22:31:57.064414 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:31:57.064983 1057 external_mini_cluster.cc:1428] Setting key d9a11a5c3acdbdfc74359cecbe220f47
I20260806 22:31:57.071722 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.061431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:35061 (local address 127.1.8.126:45483)
0806 22:31:57.061629 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.061634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.062907 (+ 1273us) server_negotiation.cc:407] Connection header received
0806 22:31:57.065234 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.065239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.065314 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.065401 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:57.066632 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.067462 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.069239 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.069660 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.069761 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.071057 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:57.071061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:57.071064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:57.071081 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:57.071108 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:57.071119 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:57.071186 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.071394 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.071470 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:31:57.072819 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.061862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:45483 (local address 127.1.8.67:35061)
0806 22:31:57.062738 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.062782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.064916 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.065634 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.065641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.065724 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:31:57.066448 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.066474 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.067629 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.067634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.069105 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.069117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.069852 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.069857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.070055 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.070783 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:31:57.070786 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:31:57.070825 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:31:57.070905 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:31:57.071203 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:31:57.071210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:31:57.071216 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.071368 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.071652 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":273,"threads_started":1}
I20260806 22:31:57.074184 29812 heartbeater.cc:347] Connected to a master server at 127.1.8.126:45483
I20260806 22:31:57.074488 29812 heartbeater.cc:464] Registering TS with master...
I20260806 22:31:57.075238 29812 heartbeater.cc:511] Master 127.1.8.126:45483 requested a full tablet report, sending...
I20260806 22:31:57.076638 29089 ts_manager.cc:194] Registered new tserver with Master: ec089260b5194b138caabd72aecd17e9 (127.1.8.67:44503)
I20260806 22:31:57.077443 29089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.67:35061
I20260806 22:31:57.080370 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:31:57.089955 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.084457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.1.8.126:45483)
0806 22:31:57.084636 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.084641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.085068 (+ 427us) server_negotiation.cc:407] Connection header received
0806 22:31:57.085194 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.085198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.085268 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.085346 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:57.085817 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.086519 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.087821 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.088210 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.088300 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.089391 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:57.089396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:57.089399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:57.089416 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:57.089445 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:57.089456 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:57.089530 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.089661 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.089731 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:31:57.093575 29089 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:31:57.100709 29089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:31:57.126929 29402 heartbeater.cc:503] Master 127.1.8.126:45483 was elected leader, sending a full tablet report...
I20260806 22:31:57.128034 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.114121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.1.8.67:44503)
0806 22:31:57.114830 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.114839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.115685 (+ 846us) server_negotiation.cc:407] Connection header received
0806 22:31:57.116241 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.116253 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.116579 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.116766 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:57.118040 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.120302 (+ 2262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.124191 (+ 3889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.126234 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.126384 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.126777 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.127493 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.127687 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":784,"threads_started":1}
I20260806 22:31:57.129197 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.113888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:44503 (local address 127.0.0.1:47620)
0806 22:31:57.115494 (+ 1606us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.115557 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.115879 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.117253 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.117264 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.117306 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:57.117772 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.117788 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.120554 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.120563 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.123898 (+ 3335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.123921 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.126968 (+ 3047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.126976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.127206 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.127267 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.128564 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.128813 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1294,"thread_start_us":1119,"threads_started":1}
I20260806 22:31:57.134153 29712 tablet_service.cc:1467] Processing CreateTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 (DEFAULT_TABLE table=test-table [id=8b2d49ed5c274e42bb40b76bb6498821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:57.135682 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.114206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34278)
0806 22:31:57.116186 (+ 1980us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.116204 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.116312 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.128430 (+ 12118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.128438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.128471 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:57.128903 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.128917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.130854 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.130859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.132498 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.132511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.134649 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.134656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.134999 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.135034 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.135125 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.135268 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1794,"mutex_wait_us":102,"thread_start_us":1639,"threads_started":1}
I20260806 22:31:57.135779 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.114723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.1.8.65:44797)
0806 22:31:57.127514 (+ 12791us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.127523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.127553 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:31:57.127663 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.127673 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.127932 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.128169 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:57.129120 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.130695 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.132697 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.134070 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.134235 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.134500 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.135226 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.135425 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12264,"thread_start_us":12451,"threads_started":1}
I20260806 22:31:57.135831 29712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84089e1c5fa4eccab84c87e85262ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:57.139454 29295 tablet_service.cc:1467] Processing CreateTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 (DEFAULT_TABLE table=test-table [id=8b2d49ed5c274e42bb40b76bb6498821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:57.140915 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84089e1c5fa4eccab84c87e85262ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:57.140990 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.116247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:38779 (local address 127.0.0.1:43192)
0806 22:31:57.126464 (+ 10217us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.126495 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.126651 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.130757 (+ 4106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.130765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.130795 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:57.131227 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.131238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.133207 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.133213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.135312 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.135326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.137039 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.137045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.140311 (+ 3266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.140347 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.140458 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.140615 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10049,"mutex_wait_us":24,"thread_start_us":10482,"threads_started":1}
I20260806 22:31:57.141108 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.116788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.1.8.66:38779)
0806 22:31:57.129963 (+ 13175us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.129972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.130001 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:31:57.130106 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.130114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.130338 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.130518 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:57.131419 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.133045 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.135538 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.136838 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.137005 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.137246 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.140549 (+ 3303us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.140741 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13005,"thread_start_us":12894,"threads_started":1}
I20260806 22:31:57.145022 29505 tablet_service.cc:1467] Processing CreateTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 (DEFAULT_TABLE table=test-table [id=8b2d49ed5c274e42bb40b76bb6498821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:31:57.146495 29505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84089e1c5fa4eccab84c87e85262ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:31:57.148768 29845 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Bootstrap starting.
I20260806 22:31:57.152155 29845 tablet_bootstrap.cc:654] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:57.153400 29845 log.cc:824] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:57.156111 29848 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Bootstrap starting.
I20260806 22:31:57.156876 29845 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: No bootstrap required, opened a new log
I20260806 22:31:57.157176 29845 ts_tablet_manager.cc:1397] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:31:57.160194 29848 tablet_bootstrap.cc:654] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:57.160938 29850 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Bootstrap starting.
I20260806 22:31:57.161581 29848 log.cc:824] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:57.163800 29850 tablet_bootstrap.cc:654] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Neither blocks nor log segments found. Creating new log.
I20260806 22:31:57.165063 29850 log.cc:824] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Log is configured to *not* fsync() on all Append() calls
I20260806 22:31:57.166652 29848 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: No bootstrap required, opened a new log
I20260806 22:31:57.166924 29848 ts_tablet_manager.cc:1397] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 22:31:57.168045 29845 raft_consensus.cc:348] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.168679 29845 raft_consensus.cc:374] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:57.168820 29845 raft_consensus.cc:729] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec089260b5194b138caabd72aecd17e9, State: Initialized, Role: FOLLOWER
I20260806 22:31:57.169400 29850 tablet_bootstrap.cc:492] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: No bootstrap required, opened a new log
I20260806 22:31:57.169708 29850 ts_tablet_manager.cc:1397] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:31:57.170364 29845 consensus_queue.cc:260] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.171551 29812 heartbeater.cc:503] Master 127.1.8.126:45483 was elected leader, sending a full tablet report...
I20260806 22:31:57.172325 29845 ts_tablet_manager.cc:1428] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260806 22:31:57.176084 29848 raft_consensus.cc:348] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.176587 29848 raft_consensus.cc:374] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:57.176697 29848 raft_consensus.cc:729] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbaec8e624ed433995bb7c995ef8f167, State: Initialized, Role: FOLLOWER
I20260806 22:31:57.177428 29848 consensus_queue.cc:260] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.179435 29850 raft_consensus.cc:348] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.179679 29848 ts_tablet_manager.cc:1428] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Time spent starting tablet: real 0.013s user 0.000s sys 0.011s
I20260806 22:31:57.179831 29850 raft_consensus.cc:374] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:31:57.179941 29850 raft_consensus.cc:729] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df494173886431a89823da2e437d832, State: Initialized, Role: FOLLOWER
I20260806 22:31:57.180634 29850 consensus_queue.cc:260] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.181766 29612 heartbeater.cc:503] Master 127.1.8.126:45483 was elected leader, sending a full tablet report...
I20260806 22:31:57.182438 29850 ts_tablet_manager.cc:1428] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260806 22:31:57.310798 29813 tablet.cc:2369] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:31:57.329540 29613 tablet.cc:2369] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:31:57.354280 29403 tablet.cc:2369] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:31:57.451956 29853 raft_consensus.cc:482] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:31:57.452384 29853 raft_consensus.cc:504] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.454190 29853 leader_election.cc:288] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbaec8e624ed433995bb7c995ef8f167 (127.1.8.65:44797), 2df494173886431a89823da2e437d832 (127.1.8.66:38779)
I20260806 22:31:57.461050 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.454641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:46051 (local address 127.1.8.65:44797)
0806 22:31:57.454853 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.454862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.454974 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:31:57.455222 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.455242 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.455344 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.455447 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:57.456084 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.457101 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.459024 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.460039 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.460218 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.460297 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.460661 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.460805 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:31:57.461074 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.454604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.1.8.67:46051)
0806 22:31:57.454838 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.454863 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.455023 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.455651 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.455661 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.455695 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:57.455906 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.455919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.457241 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.457246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.458853 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.458868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.460253 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.460259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.460452 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.460494 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.460594 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.460748 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":45}
I20260806 22:31:57.461519 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.455288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:47539 (local address 127.1.8.66:38779)
0806 22:31:57.455529 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.455535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.455863 (+ 328us) server_negotiation.cc:407] Connection header received
0806 22:31:57.455989 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.455996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.456128 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.456209 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:31:57.456800 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.457658 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.459593 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.460576 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.460690 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.460777 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.461116 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.461252 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:31:57.461542 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.455040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:38779 (local address 127.1.8.67:47539)
0806 22:31:57.455757 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.455771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:31:57.455874 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:31:57.456246 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:31:57.456251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:31:57.456276 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:31:57.456645 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.456657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.457813 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.457817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.459407 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:31:57.459424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.460746 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.460752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:31:57.460936 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.460967 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:31:57.461074 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:31:57.461209 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":1,"thread_start_us":446,"threads_started":1}
I20260806 22:31:57.462163 29327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84089e1c5fa4eccab84c87e85262ac4" candidate_uuid: "ec089260b5194b138caabd72aecd17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbaec8e624ed433995bb7c995ef8f167" is_pre_election: true
I20260806 22:31:57.462435 29534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84089e1c5fa4eccab84c87e85262ac4" candidate_uuid: "ec089260b5194b138caabd72aecd17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df494173886431a89823da2e437d832" is_pre_election: true
I20260806 22:31:57.462692 29327 raft_consensus.cc:2454] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec089260b5194b138caabd72aecd17e9 in term 0.
I20260806 22:31:57.462985 29534 raft_consensus.cc:2454] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec089260b5194b138caabd72aecd17e9 in term 0.
I20260806 22:31:57.463408 29641 leader_election.cc:302] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbaec8e624ed433995bb7c995ef8f167, ec089260b5194b138caabd72aecd17e9; no voters:
I20260806 22:31:57.463797 29853 raft_consensus.cc:2781] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:31:57.463946 29853 raft_consensus.cc:482] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:31:57.464085 29853 raft_consensus.cc:3037] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:57.466323 29853 raft_consensus.cc:504] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.466912 29853 leader_election.cc:288] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [CANDIDATE]: Term 1 election: Requested vote from peers bbaec8e624ed433995bb7c995ef8f167 (127.1.8.65:44797), 2df494173886431a89823da2e437d832 (127.1.8.66:38779)
I20260806 22:31:57.467428 29327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84089e1c5fa4eccab84c87e85262ac4" candidate_uuid: "ec089260b5194b138caabd72aecd17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbaec8e624ed433995bb7c995ef8f167"
I20260806 22:31:57.467495 29534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84089e1c5fa4eccab84c87e85262ac4" candidate_uuid: "ec089260b5194b138caabd72aecd17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df494173886431a89823da2e437d832"
I20260806 22:31:57.467670 29327 raft_consensus.cc:3037] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:57.467737 29534 raft_consensus.cc:3037] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:31:57.469964 29327 raft_consensus.cc:2454] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec089260b5194b138caabd72aecd17e9 in term 1.
I20260806 22:31:57.469975 29534 raft_consensus.cc:2454] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec089260b5194b138caabd72aecd17e9 in term 1.
I20260806 22:31:57.470409 29641 leader_election.cc:302] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df494173886431a89823da2e437d832, ec089260b5194b138caabd72aecd17e9; no voters:
I20260806 22:31:57.470718 29853 raft_consensus.cc:2781] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:31:57.471169 29853 raft_consensus.cc:686] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 1 LEADER]: Becoming Leader. State: Replica: ec089260b5194b138caabd72aecd17e9, State: Running, Role: LEADER
I20260806 22:31:57.471652 29853 consensus_queue.cc:237] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } }
I20260806 22:31:57.476286 29088 catalog_manager.cc:5495] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec089260b5194b138caabd72aecd17e9 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "ec089260b5194b138caabd72aecd17e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec089260b5194b138caabd72aecd17e9" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 44503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 } health_report { overall_health: UNKNOWN } } }
I20260806 22:31:57.507593 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.502126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.1.8.67:44503)
0806 22:31:57.502317 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.502325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.502348 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:31:57.502448 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.502452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.502559 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.502724 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:31:57.503094 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.503840 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.505316 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.505736 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.505878 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.506421 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:31:57.506950 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:31:57.507115 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.507220 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.507302 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:31:57.515978 29534 raft_consensus.cc:1261] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 1 FOLLOWER]: Refusing update from remote peer ec089260b5194b138caabd72aecd17e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:57.515978 29327 raft_consensus.cc:1261] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 1 FOLLOWER]: Refusing update from remote peer ec089260b5194b138caabd72aecd17e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:31:57.517024 29858 consensus_queue.cc:1035] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbaec8e624ed433995bb7c995ef8f167" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:31:57.517401 29853 consensus_queue.cc:1035] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df494173886431a89823da2e437d832" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:31:57.546942 29089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:57.547273 29089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:31:57.551503 29089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49ad20669dbf49f4afe1f1a252e27056: Sending DeleteTablet for 3 replicas of tablet f84089e1c5fa4eccab84c87e85262ac4
I20260806 22:31:57.552735 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:57 UTC) from {username='slave'} at 127.0.0.1:34278
I20260806 22:31:57.553186 29505 tablet_service.cc:1514] Processing DeleteTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:57 UTC) from {username='slave'} at 127.0.0.1:43192
I20260806 22:31:57.553813 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 29188
I20260806 22:31:57.557152 29712 tablet_service.cc:1514] Processing DeleteTablet for tablet f84089e1c5fa4eccab84c87e85262ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:31:57 UTC) from {username='slave'} at 127.0.0.1:47620
I20260806 22:31:57.557288 29875 tablet_replica.cc:330] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: stopping tablet replica
I20260806 22:31:57.557796 29875 raft_consensus.cc:2229] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:57.558377 29875 raft_consensus.cc:2258] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:57.560530 29875 ts_tablet_manager.cc:1905] T f84089e1c5fa4eccab84c87e85262ac4 P bbaec8e624ed433995bb7c995ef8f167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:57.564971 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.555056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.1.8.65:44797)
0806 22:31:57.555305 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.555315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.556602 (+ 1287us) server_negotiation.cc:407] Connection header received
0806 22:31:57.556806 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.556814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.556942 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.557077 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:57.557727 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.558911 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.560858 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.561491 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.561594 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.563188 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:57.563201 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:57.563217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:57.563965 (+ 748us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:57.564034 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:57.564058 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:57.564181 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.564414 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.564527 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:31:57.566341 29394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:57.566987 29880 tablet_replica.cc:330] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: stopping tablet replica
I20260806 22:31:57.567764 29880 raft_consensus.cc:2229] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:31:57.568641 29880 raft_consensus.cc:2258] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:57.572800 29880 ts_tablet_manager.cc:1905] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:57.575251 29876 tablet_replica.cc:330] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: stopping tablet replica
I20260806 22:31:57.575704 29876 raft_consensus.cc:2229] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:31:57.576311 29876 raft_consensus.cc:2258] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:31:57.583407 29880 ts_tablet_manager.cc:1918] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:57.583663 29880 log.cc:1196] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/f84089e1c5fa4eccab84c87e85262ac4
I20260806 22:31:57.584298 29880 ts_tablet_manager.cc:1939] T f84089e1c5fa4eccab84c87e85262ac4 P ec089260b5194b138caabd72aecd17e9: Deleting consensus metadata
I20260806 22:31:57.584909 29876 ts_tablet_manager.cc:1905] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:31:57.586303 29067 catalog_manager.cc:4841] TS ec089260b5194b138caabd72aecd17e9 (127.1.8.67:44503): tablet f84089e1c5fa4eccab84c87e85262ac4 (table test-table [id=8b2d49ed5c274e42bb40b76bb6498821]) successfully deleted
I20260806 22:31:57.601596 29876 ts_tablet_manager.cc:1918] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:31:57.601794 29876 log.cc:1196] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/f84089e1c5fa4eccab84c87e85262ac4
I20260806 22:31:57.602392 29876 ts_tablet_manager.cc:1939] T f84089e1c5fa4eccab84c87e85262ac4 P 2df494173886431a89823da2e437d832: Deleting consensus metadata
I20260806 22:31:57.604082 29060 catalog_manager.cc:4841] TS 2df494173886431a89823da2e437d832 (127.1.8.66:38779): tablet f84089e1c5fa4eccab84c87e85262ac4 (table test-table [id=8b2d49ed5c274e42bb40b76bb6498821]) successfully deleted
I20260806 22:31:57.718170 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 29188
W20260806 22:31:57.750452 29060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:44797 (error 108)
I20260806 22:31:57.750625 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 29406
I20260806 22:31:57.758864 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.751998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34298)
0806 22:31:57.758476 (+ 6478us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.758647 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6020,"thread_start_us":5931,"threads_started":1}
W20260806 22:31:57.759173 29060 catalog_manager.cc:4584] TS bbaec8e624ed433995bb7c995ef8f167 (127.1.8.65:44797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f84089e1c5fa4eccab84c87e85262ac4: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
I20260806 22:31:57.760524 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.751677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.1.8.66:38779)
0806 22:31:57.751895 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.751901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.751922 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:31:57.751996 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.752001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.752169 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.752319 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:57.752793 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.754245 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.756435 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.757374 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.757538 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.758758 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:57.758770 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:57.758787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:57.759669 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:57.759721 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:57.759744 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:57.759833 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.760004 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.760154 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:31:57.761147 29605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:57.812947 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.812424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34302)
0806 22:31:57.812651 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.812763 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 22:31:57.814239 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.813767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34304)
0806 22:31:57.813943 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.814047 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 22:31:57.852278 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 29406
I20260806 22:31:57.874843 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.874394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34318)
0806 22:31:57.874612 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.874693 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 22:31:57.875978 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.875627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34322)
0806 22:31:57.875775 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.875849 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:31:57.881526 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 29616
I20260806 22:31:57.889470 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.882281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.1.8.67:44503)
0806 22:31:57.882446 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:31:57.882450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:31:57.882467 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:31:57.882671 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:31:57.882675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:31:57.882762 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:31:57.882843 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:31:57.883355 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.884246 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.886155 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:31:57.886597 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:31:57.886743 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:31:57.887912 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:31:57.887920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:31:57.887935 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:31:57.888691 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:31:57.888742 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:31:57.888762 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:31:57.888845 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:31:57.889044 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 22:31:57.889123 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:31:57.890195 29807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:57.957075 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.956504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34328)
0806 22:31:57.956747 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.956908 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 22:31:57.958272 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:31:57.957824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:44797 (local address 127.0.0.1:34338)
0806 22:31:57.958055 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:31:57.958136 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:44797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 22:31:58.012382 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 29616
I20260806 22:31:58.039012 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 29032
I20260806 22:31:58.040166 29142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:31:58.147545 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 29032
2026-08-06T22:31:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3125 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:31:58 dist-test-slave-7vjv krb5kdc[29892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:31:58 dist-test-slave-7vjv krb5kdc[29892](info): set up 2 sockets
Aug 06 22:31:58 dist-test-slave-7vjv krb5kdc[29892](info): commencing operation
krb5kdc: starting...
W20260806 22:31:59.519488 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:31:59 dist-test-slave-7vjv krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:31:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:31:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:31:59.856806 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34427
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:00.206840 29913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:00.207154 29913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:00.207343 29913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:00.214700 29913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:00.214833 29913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:00.214924 29913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:00.215015 29913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:00.215106 29913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:00.225325 29913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:00.226786 29913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:00.228212 29913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:00.240955 29920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:00.241042 29930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:00.240993 29924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:00.241756 29913 server_base.cc:1034] running on GCE node
I20260806 22:32:00.242674 29913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:00.244287 29913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:00.245555 29913 hybrid_clock.cc:648] HybridClock initialized: now 1786055520245511 us; error 51 us; skew 500 ppm
I20260806 22:32:00.245949 29913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:00 dist-test-slave-7vjv krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:00.253216 29913 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:00.254961 29913 webserver.cc:460] Webserver started at http://127.1.8.126:36341/ using document root <none> and password file <none>
I20260806 22:32:00.255733 29913 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:00.255885 29913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:00.256314 29913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:00.259497 29913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "47ce044ad9d641d9b50747ccab7490bc"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "8110a12e7815933046b4c0885198114d"
server_key_iv: "d43d542aedcf37bf11349d6362175086"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.260386 29913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "47ce044ad9d641d9b50747ccab7490bc"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "8110a12e7815933046b4c0885198114d"
server_key_iv: "d43d542aedcf37bf11349d6362175086"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.265928 29913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:00.270197 29943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:00.272346 29913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:32:00.272511 29913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "47ce044ad9d641d9b50747ccab7490bc"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "8110a12e7815933046b4c0885198114d"
server_key_iv: "d43d542aedcf37bf11349d6362175086"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.272710 29913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:00.296648 29913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:00.301455 29913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:00.301735 29913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:00.321545 29913 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34427
I20260806 22:32:00.321570 30026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34427 every 8 connection(s)
I20260806 22:32:00.323048 29913 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:00.326938 30027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:00.329469 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 29913
I20260806 22:32:00.329679 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:00.330262 1057 external_mini_cluster.cc:1428] Setting key ab3a8b04523fb91a6c9eeaa27bb23b67
I20260806 22:32:00.341909 30027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: Bootstrap starting.
I20260806 22:32:00.345490 30027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:00.346551 30027 log.cc:824] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:32:00 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:00.349752 30027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: No bootstrap required, opened a new log
I20260806 22:32:00.356488 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:00.332465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.1.8.126:34427)
0806 22:32:00.333416 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:00.333429 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:00.333475 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:32:00.335545 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:00.335589 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:00.336213 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:00.336725 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:00.338925 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.341039 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.343195 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.343943 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.344223 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:00.348728 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:00.348739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:00.348755 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:00.348792 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:00.352050 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:00.352892 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:00.352901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:00.352907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:00.353092 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:00.353513 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:00.353520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:00.353528 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:00.354069 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:00.354426 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:00.354877 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 22:32:00.355351 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":440,"threads_started":1}
I20260806 22:32:00.358281 30027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } }
I20260806 22:32:00.358779 30027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:00.358892 30027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ce044ad9d641d9b50747ccab7490bc, State: Initialized, Role: FOLLOWER
I20260806 22:32:00.359583 30027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } }
I20260806 22:32:00.359886 30027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:00.359998 30027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:00.360188 30027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:00.362670 30027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } }
I20260806 22:32:00.363332 30027 leader_election.cc:302] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47ce044ad9d641d9b50747ccab7490bc; no voters:
I20260806 22:32:00.363873 30027 leader_election.cc:288] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:00.364043 30035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:00.364832 30035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [term 1 LEADER]: Becoming Leader. State: Replica: 47ce044ad9d641d9b50747ccab7490bc, State: Running, Role: LEADER
I20260806 22:32:00.365485 30035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } }
I20260806 22:32:00.366130 30027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:00.368810 30038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47ce044ad9d641d9b50747ccab7490bc. Latest consensus state: current_term: 1 leader_uuid: "47ce044ad9d641d9b50747ccab7490bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } } }
I20260806 22:32:00.369297 30038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:00.372365 30036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47ce044ad9d641d9b50747ccab7490bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ce044ad9d641d9b50747ccab7490bc" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34427 } } }
I20260806 22:32:00.372776 30036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:00.374259 30045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:00.379439 30045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:00.388307 30045 catalog_manager.cc:1347] Generated new cluster ID: 7b2a33ec294f42bd9f9de8080c8057ea
I20260806 22:32:00.388430 30045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:00.405216 30045 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:00.407169 30045 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:00.418480 30045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: Generated new TSK 0
I20260806 22:32:00.419574 30045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:32:00.499142 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:00.848583 30067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:00.848850 30067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:00.849037 30067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:00.856336 30067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:00.856483 30067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:00.856655 30067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:32:00.866580 30067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:00.867993 30067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:00.869344 30067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:00.883402 30078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:00.883380 30076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:00.883351 30073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:00.884150 30067 server_base.cc:1034] running on GCE node
I20260806 22:32:00.884716 30067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:00.885561 30067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:00.886770 30067 hybrid_clock.cc:648] HybridClock initialized: now 1786055520886727 us; error 40 us; skew 500 ppm
I20260806 22:32:00.887028 30067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:00 dist-test-slave-7vjv krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:00.890223 30067 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:32:00.891631 30067 webserver.cc:460] Webserver started at http://127.1.8.65:44201/ using document root <none> and password file <none>
I20260806 22:32:00.892392 30067 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:00.892537 30067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:00.892858 30067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:00.895797 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "3abe0b195dbe47338ace072f5c99ad6c"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "d0fa934e831f1d95f4940cb108edcf3d"
server_key_iv: "1bdd418dcf5cfc94127e80f2ac84463c"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.896674 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "3abe0b195dbe47338ace072f5c99ad6c"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "d0fa934e831f1d95f4940cb108edcf3d"
server_key_iv: "1bdd418dcf5cfc94127e80f2ac84463c"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.901886 30067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:00.905375 30094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:00.906800 30067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:00.906982 30067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "3abe0b195dbe47338ace072f5c99ad6c"
format_stamp: "Formatted at 2026-08-06 22:32:00 on dist-test-slave-7vjv"
server_key: "d0fa934e831f1d95f4940cb108edcf3d"
server_key_iv: "1bdd418dcf5cfc94127e80f2ac84463c"
server_key_version: "encryptionkey@0"
I20260806 22:32:00.907202 30067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:00.921062 30067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:00.925036 30067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:00.925294 30067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:00.926219 30067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:00.927446 30067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:00.927520 30067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:00.927618 30067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:00.927664 30067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:00.960562 30067 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:43713
I20260806 22:32:00.960650 30255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:43713 every 8 connection(s)
I20260806 22:32:00.961896 30067 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:32:00.962877 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 30067
I20260806 22:32:00.963066 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:32:00.963537 1057 external_mini_cluster.cc:1428] Setting key fad0b964a93537bfdebe269b22c7e517
Aug 06 22:32:00 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:00.983438 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:00.964702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:35899 (local address 127.1.8.126:34427)
0806 22:32:00.965183 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:00.965189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:00.966236 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 22:32:00.968046 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:00.968054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:00.968190 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:00.968302 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:00.969760 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.970854 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.972448 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.973152 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.973296 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:00.977352 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:00.977357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:00.977360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:00.977391 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:00.980220 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:00.980925 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:00.980932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:00.980937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:00.981107 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:00.981518 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:00.981526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:00.981531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:00.981914 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:00.982081 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:00.982695 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 22:32:00.982950 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
I20260806 22:32:00.983995 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:00.965202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34427 (local address 127.1.8.65:35899)
0806 22:32:00.966070 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:00.966118 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:00.967797 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:00.968505 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:00.968512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:00.969035 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:00.969590 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:00.969606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.971008 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.971011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:00.972309 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:00.972321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:00.973362 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:00.973366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:00.973559 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:00.974338 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:00.974361 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:00.977146 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:00.980390 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:00.980396 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:00.980408 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:00.980806 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:00.981222 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:00.981225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:00.981228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:00.981401 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:00.982036 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:00.982041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:00.982337 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 22:32:00.982550 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 22:32:00.982904 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":376,"threads_started":1}
I20260806 22:32:00.985688 30256 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34427
I20260806 22:32:00.986006 30256 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:00.986742 30256 heartbeater.cc:511] Master 127.1.8.126:34427 requested a full tablet report, sending...
I20260806 22:32:00.989305 29970 ts_manager.cc:194] Registered new tserver with Master: 3abe0b195dbe47338ace072f5c99ad6c (127.1.8.65:43713)
I20260806 22:32:00.990775 29970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:35899
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:32:01.035210 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:01.378109 30262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:01.378430 30262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:01.378608 30262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:01.385800 30262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:01.385947 30262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:01.386157 30262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:32:01.396350 30262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:01.397846 30262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:01.399278 30262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:01.413774 30268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:01.413856 30270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:01.413756 30273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:01.414862 30262 server_base.cc:1034] running on GCE node
I20260806 22:32:01.415495 30262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:01.416628 30262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:01.417861 30262 hybrid_clock.cc:648] HybridClock initialized: now 1786055521417768 us; error 80 us; skew 500 ppm
I20260806 22:32:01.418198 30262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:01 dist-test-slave-7vjv krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:01.421651 30262 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:32:01.423364 30262 webserver.cc:460] Webserver started at http://127.1.8.66:41787/ using document root <none> and password file <none>
I20260806 22:32:01.424176 30262 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:01.424330 30262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:01.424669 30262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:01.427707 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "c37b5cb770fe4f3ab590d6c7a0565e56"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "68ad514e620d4a7b2356d461c86bcf74"
server_key_iv: "7b660597a8529a7032e3e443e2339944"
server_key_version: "encryptionkey@0"
I20260806 22:32:01.428601 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "c37b5cb770fe4f3ab590d6c7a0565e56"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "68ad514e620d4a7b2356d461c86bcf74"
server_key_iv: "7b660597a8529a7032e3e443e2339944"
server_key_version: "encryptionkey@0"
I20260806 22:32:01.433861 30262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:01.437377 30283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:01.438681 30262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:01.438866 30262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "c37b5cb770fe4f3ab590d6c7a0565e56"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "68ad514e620d4a7b2356d461c86bcf74"
server_key_iv: "7b660597a8529a7032e3e443e2339944"
server_key_version: "encryptionkey@0"
I20260806 22:32:01.439090 30262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:01.470496 30262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:01.475397 30262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:01.475747 30262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:01.476951 30262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:01.478505 30262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:01.478621 30262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:01.478785 30262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:01.478885 30262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:01.515864 30262 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:38063
I20260806 22:32:01.515959 30446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:38063 every 8 connection(s)
I20260806 22:32:01.517277 30262 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:32:01.526311 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 30262
I20260806 22:32:01.526526 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:32:01.527182 1057 external_mini_cluster.cc:1428] Setting key 42877b6448276051097cfe4be241e55e
Aug 06 22:32:01 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:01.538353 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:01.519942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:45369 (local address 127.1.8.126:34427)
0806 22:32:01.520466 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:01.520470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:01.521416 (+ 946us) server_negotiation.cc:407] Connection header received
0806 22:32:01.523277 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:01.523284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:01.523379 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:01.523505 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:01.525102 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:01.525837 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:01.528019 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:01.528583 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:01.528724 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:01.532368 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:01.532373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:01.532376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:01.532410 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:01.535153 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:01.535920 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:01.535924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:01.535927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:01.536057 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:01.536576 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:01.536583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:01.536591 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:01.536850 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:01.537016 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:01.537716 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 22:32:01.537948 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":301,"threads_started":1}
I20260806 22:32:01.539351 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:01.520541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34427 (local address 127.1.8.66:45369)
0806 22:32:01.521246 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:01.521284 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:01.523005 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:01.523757 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:01.523765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:01.524361 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:01.524914 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:01.524932 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:01.526016 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:01.526040 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:01.527862 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:01.527875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:01.528832 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:01.528840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:01.529089 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:01.529811 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:01.529835 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:01.532068 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:01.535307 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:01.535316 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:01.535334 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:01.535778 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:01.536229 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:01.536236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:01.536240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:01.536450 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:01.536986 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:01.536993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:01.537348 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 22:32:01.537620 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 22:32:01.538163 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":287,"threads_started":1}
I20260806 22:32:01.541007 30447 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34427
I20260806 22:32:01.541311 30447 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:01.542085 30447 heartbeater.cc:511] Master 127.1.8.126:34427 requested a full tablet report, sending...
I20260806 22:32:01.543762 29970 ts_manager.cc:194] Registered new tserver with Master: c37b5cb770fe4f3ab590d6c7a0565e56 (127.1.8.66:38063)
I20260806 22:32:01.544677 29970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:45369
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:32:01.600656 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:01.944830 30456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:01.945106 30456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:01.945200 30456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:01.952073 30456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:01.952246 30456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:01.952358 30456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:32:01.961750 30456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:34427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:01.962911 30456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:01.964244 30456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:01.977803 30468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:01.977777 30466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:01.978281 30465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:01.978730 30456 server_base.cc:1034] running on GCE node
I20260806 22:32:01.979310 30456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:01.980234 30456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:01.981434 30456 hybrid_clock.cc:648] HybridClock initialized: now 1786055521981398 us; error 28 us; skew 500 ppm
I20260806 22:32:01.981753 30456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:01 dist-test-slave-7vjv krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:01.985056 30456 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:32:01.986543 30456 webserver.cc:460] Webserver started at http://127.1.8.67:35175/ using document root <none> and password file <none>
I20260806 22:32:01.987273 30456 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:01.987425 30456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:01.987790 30456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:01.990783 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "77294d9234b5404695003864df5176c6"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "ae433a0539987c53baf504dda93bb0ee"
server_key_iv: "59b2792a6ffbcc50306af2647b0daa6d"
server_key_version: "encryptionkey@0"
I20260806 22:32:01.991590 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "77294d9234b5404695003864df5176c6"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "ae433a0539987c53baf504dda93bb0ee"
server_key_iv: "59b2792a6ffbcc50306af2647b0daa6d"
server_key_version: "encryptionkey@0"
I20260806 22:32:01.993798 30256 heartbeater.cc:503] Master 127.1.8.126:34427 was elected leader, sending a full tablet report...
I20260806 22:32:01.997308 30456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 22:32:02.000833 30483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:02.002161 30456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:02.002350 30456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "77294d9234b5404695003864df5176c6"
format_stamp: "Formatted at 2026-08-06 22:32:01 on dist-test-slave-7vjv"
server_key: "ae433a0539987c53baf504dda93bb0ee"
server_key_iv: "59b2792a6ffbcc50306af2647b0daa6d"
server_key_version: "encryptionkey@0"
I20260806 22:32:02.002583 30456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:02.015901 30456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:02.020550 30456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:02.021039 30456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:02.022091 30456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:02.023561 30456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:02.023679 30456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:02.023842 30456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:02.023931 30456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:02.061105 30456 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:46713
I20260806 22:32:02.061141 30662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:46713 every 8 connection(s)
I20260806 22:32:02.062446 30456 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:32:02.064625 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 30456
I20260806 22:32:02.064823 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:32:02.065346 1057 external_mini_cluster.cc:1428] Setting key 8469102f13b2567990df2ef783119ac4
Aug 06 22:32:02 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:02.083591 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.065384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:49221 (local address 127.1.8.126:34427)
0806 22:32:02.065983 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.065990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.067154 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 22:32:02.069392 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.069397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.069483 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.069585 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:02.071162 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.071907 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.073442 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.073923 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.074017 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.077461 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:02.077466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:02.077469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:02.077500 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:02.080814 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.081599 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.081603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.081605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.081736 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.082123 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.082126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.082128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.082380 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:02.082494 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.083128 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.083310 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":231,"threads_started":1}
I20260806 22:32:02.084586 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.066057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:34427 (local address 127.1.8.67:49221)
0806 22:32:02.066971 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.067020 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.069141 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.069787 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.069794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.070318 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:02.070883 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.070900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.072032 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.072035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.073306 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.073318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.074131 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.074136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.074326 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.074979 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:02.075001 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:02.077286 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:02.080957 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:02.080965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:02.080983 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:02.081475 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:02.081823 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:02.081826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:02.081828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:02.082023 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:02.082488 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:02.082494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:02.082802 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.083014 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.083399 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":366,"threads_started":1}
I20260806 22:32:02.085861 30663 heartbeater.cc:347] Connected to a master server at 127.1.8.126:34427
I20260806 22:32:02.086127 30663 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:02.086787 30663 heartbeater.cc:511] Master 127.1.8.126:34427 requested a full tablet report, sending...
I20260806 22:32:02.088193 29970 ts_manager.cc:194] Registered new tserver with Master: 77294d9234b5404695003864df5176c6 (127.1.8.67:46713)
I20260806 22:32:02.089016 29970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:49221
I20260806 22:32:02.093107 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:32:02.110412 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.097388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46972 (local address 127.1.8.126:34427)
0806 22:32:02.097554 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.097558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.098063 (+ 505us) server_negotiation.cc:407] Connection header received
0806 22:32:02.098239 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.098243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.098300 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.098376 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:02.099878 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.100659 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.101953 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.102324 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.102411 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.105025 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:02.105029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:02.105032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:02.105062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:02.107698 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.108520 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.108523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.108526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.108662 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.109179 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.109182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.109184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.109481 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:02.109684 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.110022 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.110172 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:32:02.114385 29970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:32:02.121400 29970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:32:02.144584 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.133940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.1.8.67:46713)
0806 22:32:02.135211 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.135217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.135268 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:32:02.135379 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.135386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.135658 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.135889 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:02.136965 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.138682 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.141825 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.143266 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.143401 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.143640 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.143972 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.144197 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":964,"threads_started":1}
I20260806 22:32:02.144723 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.133592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:46713 (local address 127.0.0.1:36672)
0806 22:32:02.134738 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.134782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.134942 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.136140 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.136151 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.136194 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:02.136721 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.136738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.138880 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.138886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.141643 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.141657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.143478 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.143484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.143702 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.143753 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.143896 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.144182 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":337,"thread_start_us":759,"threads_started":1}
I20260806 22:32:02.145859 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.135467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.1.8.65:43713)
0806 22:32:02.136234 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.136246 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.136962 (+ 716us) server_negotiation.cc:407] Connection header received
0806 22:32:02.137084 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.137094 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.137330 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.137533 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:02.138399 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.140085 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.142659 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.144053 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.144250 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.144480 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.145385 (+ 905us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.145541 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":342,"threads_started":1}
I20260806 22:32:02.145906 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.135157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:43713 (local address 127.0.0.1:54942)
0806 22:32:02.136646 (+ 1489us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.136673 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.136830 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.137790 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.137795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.137821 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:02.138194 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.138207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.140256 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.140264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.142480 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.142499 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.144644 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.144653 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.145187 (+ 534us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.145221 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.145338 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.145463 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1314,"mutex_wait_us":30,"thread_start_us":1144,"threads_started":1}
I20260806 22:32:02.148381 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.137256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:38063 (local address 127.0.0.1:57560)
0806 22:32:02.138602 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.138627 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.138769 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.139788 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.139795 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.139821 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:02.140414 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.140430 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.142376 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.142382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.144401 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.144415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.147572 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.147579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.147757 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.147802 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.147911 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.148035 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"thread_start_us":1043,"threads_started":1}
I20260806 22:32:02.148502 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.136918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.1.8.66:38063)
0806 22:32:02.139300 (+ 2382us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.139307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.139331 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:32:02.139409 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.139415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.139572 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.139802 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:02.140658 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.142178 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.144599 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.147348 (+ 2749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.147492 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.147665 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.147979 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.148174 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2228,"thread_start_us":2107,"threads_started":1}
I20260806 22:32:02.149348 30567 tablet_service.cc:1467] Processing CreateTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c (DEFAULT_TABLE table=test-table [id=450e17aa22934bd58ca50e52dc5fc089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:02.149585 30161 tablet_service.cc:1467] Processing CreateTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c (DEFAULT_TABLE table=test-table [id=450e17aa22934bd58ca50e52dc5fc089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:02.150985 30161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb3a1171be64b87a30b6c861eb9cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:02.151351 30567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb3a1171be64b87a30b6c861eb9cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:02.152138 30355 tablet_service.cc:1467] Processing CreateTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c (DEFAULT_TABLE table=test-table [id=450e17aa22934bd58ca50e52dc5fc089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:02.153488 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb3a1171be64b87a30b6c861eb9cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:02.163763 30695 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Bootstrap starting.
I20260806 22:32:02.166538 30696 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Bootstrap starting.
I20260806 22:32:02.168331 30695 tablet_bootstrap.cc:654] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:02.168820 30697 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Bootstrap starting.
I20260806 22:32:02.169808 30695 log.cc:824] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:02.170078 30696 tablet_bootstrap.cc:654] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:02.171362 30696 log.cc:824] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:02.172097 30697 tablet_bootstrap.cc:654] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:02.173223 30697 log.cc:824] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:02.173591 30695 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: No bootstrap required, opened a new log
I20260806 22:32:02.173877 30695 ts_tablet_manager.cc:1397] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 22:32:02.174880 30696 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: No bootstrap required, opened a new log
I20260806 22:32:02.175136 30696 ts_tablet_manager.cc:1397] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260806 22:32:02.178402 30697 tablet_bootstrap.cc:492] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: No bootstrap required, opened a new log
I20260806 22:32:02.178665 30697 ts_tablet_manager.cc:1397] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 22:32:02.184777 30695 raft_consensus.cc:348] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.185251 30695 raft_consensus.cc:374] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:02.185348 30695 raft_consensus.cc:729] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abe0b195dbe47338ace072f5c99ad6c, State: Initialized, Role: FOLLOWER
I20260806 22:32:02.185788 30696 raft_consensus.cc:348] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.185878 30697 raft_consensus.cc:348] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.186219 30696 raft_consensus.cc:374] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:02.186110 30695 consensus_queue.cc:260] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.186337 30696 raft_consensus.cc:729] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77294d9234b5404695003864df5176c6, State: Initialized, Role: FOLLOWER
I20260806 22:32:02.186439 30697 raft_consensus.cc:374] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:02.186563 30697 raft_consensus.cc:729] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37b5cb770fe4f3ab590d6c7a0565e56, State: Initialized, Role: FOLLOWER
I20260806 22:32:02.187002 30696 consensus_queue.cc:260] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.187175 30697 consensus_queue.cc:260] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.188568 30695 ts_tablet_manager.cc:1428] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260806 22:32:02.189597 30663 heartbeater.cc:503] Master 127.1.8.126:34427 was elected leader, sending a full tablet report...
I20260806 22:32:02.190090 30447 heartbeater.cc:503] Master 127.1.8.126:34427 was elected leader, sending a full tablet report...
I20260806 22:32:02.190188 30696 ts_tablet_manager.cc:1428] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 22:32:02.192111 30697 ts_tablet_manager.cc:1428] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
W20260806 22:32:02.214557 30257 tablet.cc:2369] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:32:02.269490 30449 tablet.cc:2369] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:32:02.314224 30665 tablet.cc:2369] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:02.344957 30708 raft_consensus.cc:482] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:32:02.345301 30708 raft_consensus.cc:504] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.347035 30708 leader_election.cc:288] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3abe0b195dbe47338ace072f5c99ad6c (127.1.8.65:43713), c37b5cb770fe4f3ab590d6c7a0565e56 (127.1.8.66:38063)
I20260806 22:32:02.352762 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.347363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:43713 (local address 127.1.8.67:40031)
0806 22:32:02.347608 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.347629 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.347790 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.348381 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.348386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.348419 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:02.348587 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.348594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.349570 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.349575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.350840 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.350852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.351993 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.351999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.352222 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.352250 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.352358 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.352488 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:32:02.352767 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.347520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:40031 (local address 127.1.8.65:43713)
0806 22:32:02.347756 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.347763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.347788 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:32:02.347948 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.347954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.348049 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.348194 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:02.348730 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.349439 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.351000 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.351816 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.351975 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.352083 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.352388 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.352508 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:32:02.353680 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.347995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:45363 (local address 127.1.8.66:38063)
0806 22:32:02.348268 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.348276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.348618 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:32:02.348818 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.348823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.348892 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.348986 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:02.349495 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.350189 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.351870 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.352747 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.352896 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.352990 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.353333 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.353427 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 22:32:02.353713 30710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.347764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:38063 (local address 127.1.8.67:45363)
0806 22:32:02.348508 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:02.348522 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:02.348672 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:02.349003 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:02.349007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:02.349031 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:02.349346 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.349354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.350333 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.350338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.351700 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:02.351711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.352950 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.352956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:02.353124 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.353185 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:32:02.353284 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:32:02.353410 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":438,"threads_started":1}
I20260806 22:32:02.354079 30190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb3a1171be64b87a30b6c861eb9cf6c" candidate_uuid: "77294d9234b5404695003864df5176c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abe0b195dbe47338ace072f5c99ad6c" is_pre_election: true
I20260806 22:32:02.354529 30190 raft_consensus.cc:2454] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77294d9234b5404695003864df5176c6 in term 0.
I20260806 22:32:02.355136 30485 leader_election.cc:302] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abe0b195dbe47338ace072f5c99ad6c, 77294d9234b5404695003864df5176c6; no voters:
I20260806 22:32:02.355547 30708 raft_consensus.cc:2781] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:32:02.355705 30708 raft_consensus.cc:482] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:32:02.355840 30708 raft_consensus.cc:3037] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:02.355749 30386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb3a1171be64b87a30b6c861eb9cf6c" candidate_uuid: "77294d9234b5404695003864df5176c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" is_pre_election: true
I20260806 22:32:02.356364 30386 raft_consensus.cc:2454] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77294d9234b5404695003864df5176c6 in term 0.
I20260806 22:32:02.358289 30708 raft_consensus.cc:504] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.358856 30708 leader_election.cc:288] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [CANDIDATE]: Term 1 election: Requested vote from peers 3abe0b195dbe47338ace072f5c99ad6c (127.1.8.65:43713), c37b5cb770fe4f3ab590d6c7a0565e56 (127.1.8.66:38063)
I20260806 22:32:02.359341 30190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb3a1171be64b87a30b6c861eb9cf6c" candidate_uuid: "77294d9234b5404695003864df5176c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abe0b195dbe47338ace072f5c99ad6c"
I20260806 22:32:02.359419 30386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb3a1171be64b87a30b6c861eb9cf6c" candidate_uuid: "77294d9234b5404695003864df5176c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56"
I20260806 22:32:02.359548 30190 raft_consensus.cc:3037] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:02.359685 30386 raft_consensus.cc:3037] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:02.361898 30386 raft_consensus.cc:2454] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77294d9234b5404695003864df5176c6 in term 1.
I20260806 22:32:02.361898 30190 raft_consensus.cc:2454] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77294d9234b5404695003864df5176c6 in term 1.
I20260806 22:32:02.362294 30485 leader_election.cc:302] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77294d9234b5404695003864df5176c6, c37b5cb770fe4f3ab590d6c7a0565e56; no voters:
I20260806 22:32:02.362566 30708 raft_consensus.cc:2781] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:02.363030 30708 raft_consensus.cc:686] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 1 LEADER]: Becoming Leader. State: Replica: 77294d9234b5404695003864df5176c6, State: Running, Role: LEADER
I20260806 22:32:02.363602 30708 consensus_queue.cc:237] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } }
I20260806 22:32:02.368362 29970 catalog_manager.cc:5495] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77294d9234b5404695003864df5176c6 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "77294d9234b5404695003864df5176c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77294d9234b5404695003864df5176c6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 } health_report { overall_health: UNKNOWN } } }
I20260806 22:32:02.384953 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.379227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.1.8.67:46713)
0806 22:32:02.379441 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.379447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.379560 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:32:02.379815 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.379819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.379893 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.380046 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:32:02.380489 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.381207 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.382639 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.383019 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.383114 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.383667 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:32:02.384249 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:32:02.384452 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.384593 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.384669 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:32:02.393208 30190 raft_consensus.cc:1261] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 1 FOLLOWER]: Refusing update from remote peer 77294d9234b5404695003864df5176c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:02.393181 30386 raft_consensus.cc:1261] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 1 FOLLOWER]: Refusing update from remote peer 77294d9234b5404695003864df5176c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:02.394271 30711 consensus_queue.cc:1035] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37b5cb770fe4f3ab590d6c7a0565e56" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:02.394707 30708 consensus_queue.cc:1035] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abe0b195dbe47338ace072f5c99ad6c" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:02.406775 30717 mvcc.cc:205] Tried to move back new op lower bound from 7315683419712212992 to 7315683419605856256. Current Snapshot: MvccSnapshot[applied={T|T < 7315683419712212992}]
I20260806 22:32:02.408766 30718 mvcc.cc:205] Tried to move back new op lower bound from 7315683419712212992 to 7315683419605856256. Current Snapshot: MvccSnapshot[applied={T|T < 7315683419712212992}]
I20260806 22:32:02.420495 29969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:02.420830 29969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:02.424719 29969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47ce044ad9d641d9b50747ccab7490bc: Sending DeleteTablet for 3 replicas of tablet ecb3a1171be64b87a30b6c861eb9cf6c
I20260806 22:32:02.426023 30161 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:02 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:54942
I20260806 22:32:02.426033 30355 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:02 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:57560
I20260806 22:32:02.426952 30733 tablet_replica.cc:330] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: stopping tablet replica
I20260806 22:32:02.426920 30567 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb3a1171be64b87a30b6c861eb9cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:02 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:36672
I20260806 22:32:02.427584 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 30067
I20260806 22:32:02.427527 30733 raft_consensus.cc:2229] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:02.428339 30733 raft_consensus.cc:2258] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:02.430945 30733 ts_tablet_manager.cc:1905] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:02.439476 30733 ts_tablet_manager.cc:1918] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:02.439785 30733 log.cc:1196] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/ecb3a1171be64b87a30b6c861eb9cf6c
I20260806 22:32:02.440492 30733 ts_tablet_manager.cc:1939] T ecb3a1171be64b87a30b6c861eb9cf6c P 3abe0b195dbe47338ace072f5c99ad6c: Deleting consensus metadata
I20260806 22:32:02.442586 29944 catalog_manager.cc:4841] TS 3abe0b195dbe47338ace072f5c99ad6c (127.1.8.65:43713): tablet ecb3a1171be64b87a30b6c861eb9cf6c (table test-table [id=450e17aa22934bd58ca50e52dc5fc089]) successfully deleted
I20260806 22:32:02.444747 30734 tablet_replica.cc:330] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: stopping tablet replica
I20260806 22:32:02.445007 30735 tablet_replica.cc:330] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: stopping tablet replica
I20260806 22:32:02.445350 30734 raft_consensus.cc:2229] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:02.445426 30735 raft_consensus.cc:2229] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:32:02.445976 30734 raft_consensus.cc:2258] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:02.446105 30735 raft_consensus.cc:2258] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:02.448338 30734 ts_tablet_manager.cc:1905] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:02.448504 30735 ts_tablet_manager.cc:1905] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:32:02 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:32:02.455749 30734 ts_tablet_manager.cc:1918] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:02.456001 30734 log.cc:1196] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/ecb3a1171be64b87a30b6c861eb9cf6c
I20260806 22:32:02.456313 30735 ts_tablet_manager.cc:1918] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:02.456492 30735 log.cc:1196] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/ecb3a1171be64b87a30b6c861eb9cf6c
I20260806 22:32:02.456671 30734 ts_tablet_manager.cc:1939] T ecb3a1171be64b87a30b6c861eb9cf6c P c37b5cb770fe4f3ab590d6c7a0565e56: Deleting consensus metadata
I20260806 22:32:02.457069 30735 ts_tablet_manager.cc:1939] T ecb3a1171be64b87a30b6c861eb9cf6c P 77294d9234b5404695003864df5176c6: Deleting consensus metadata
I20260806 22:32:02.458626 29944 catalog_manager.cc:4841] TS c37b5cb770fe4f3ab590d6c7a0565e56 (127.1.8.66:38063): tablet ecb3a1171be64b87a30b6c861eb9cf6c (table test-table [id=450e17aa22934bd58ca50e52dc5fc089]) successfully deleted
I20260806 22:32:02.458951 29944 catalog_manager.cc:4841] TS 77294d9234b5404695003864df5176c6 (127.1.8.67:46713): tablet ecb3a1171be64b87a30b6c861eb9cf6c (table test-table [id=450e17aa22934bd58ca50e52dc5fc089]) successfully deleted
I20260806 22:32:02.462316 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.430919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.1.8.65:43713)
0806 22:32:02.431147 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.431156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.441674 (+ 10518us) server_negotiation.cc:407] Connection header received
0806 22:32:02.442241 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.442250 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.442393 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.442485 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:02.444854 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.445942 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.447906 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.448587 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.448686 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.454991 (+ 6305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:02.455009 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:02.455030 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:02.455079 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:02.459328 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.460065 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.460081 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.460083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.460321 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.460717 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.460722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.460726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.461083 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:02.461409 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.461720 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.461937 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:32:02.463022 30250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:02.597431 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 30067
I20260806 22:32:02.622857 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 30262
Aug 06 22:32:02 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:32:02.638792 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.623779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.1.8.66:38063)
0806 22:32:02.623983 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.623989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.624010 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:32:02.624088 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.624116 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.624209 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.624325 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:02.625658 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.626406 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.627682 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.628068 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.628390 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.632652 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:02.632667 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:02.632683 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:02.632721 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:02.635689 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.636575 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.636590 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.636592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.636728 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.637120 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.637127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.637133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.637527 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:02.637864 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.638178 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.638416 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:32:02.639496 30442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:02.731410 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 30262
I20260806 22:32:02.765168 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 30456
Aug 06 22:32:02 dist-test-slave-7vjv krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:32:02.783027 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:02.766027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.1.8.67:46713)
0806 22:32:02.766267 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:02.766273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:02.766293 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:32:02.766450 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:02.766455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:02.766548 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:02.766690 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:02.768088 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.768929 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.770211 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:02.770698 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:02.770842 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:02.775568 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:02.775587 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:02.775607 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:02.775657 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:02.779765 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.780740 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.780751 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.780756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.781027 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:02.781377 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:02.781388 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:02.781391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:02.781811 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:02.782189 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:02.782352 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:32:02.782572 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:32:02.783870 30656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:02.907627 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 30456
I20260806 22:32:02.935750 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 29913
I20260806 22:32:02.936967 30020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:03.051291 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 29913
2026-08-06T22:32:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4904 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:03Z Disabled control of system clock
I20260806 22:32:03.152141 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38287
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:34643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:32:03.495460 30746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:03.495924 30746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:03.496133 30746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:03.503418 30746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:03.503510 30746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:03.503556 30746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:03.503610 30746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:03.503654 30746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:03.512475 30746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:03.513644 30746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:03.515376 30746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:03.526757 30753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:03.526895 30752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:03.526815 30756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:03.528014 30746 server_base.cc:1034] running on GCE node
I20260806 22:32:03.528793 30746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:03.530018 30746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:03.531242 30746 hybrid_clock.cc:648] HybridClock initialized: now 1786055523531206 us; error 46 us; skew 500 ppm
I20260806 22:32:03.531617 30746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:03.533898 30746 webserver.cc:460] Webserver started at http://127.1.8.126:33203/ using document root <none> and password file <none>
I20260806 22:32:03.534576 30746 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:03.534715 30746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:03.535050 30746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:03.537998 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "3470018af2c94d3091be7aa577691e45"
format_stamp: "Formatted at 2026-08-06 22:32:03 on dist-test-slave-7vjv"
server_key: "a596d937b5a18edc3e126176b3c2b66b"
server_key_iv: "c86791c9f0b4393fb2ba137694095b1a"
server_key_version: "encryptionkey@0"
I20260806 22:32:03.538780 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "3470018af2c94d3091be7aa577691e45"
format_stamp: "Formatted at 2026-08-06 22:32:03 on dist-test-slave-7vjv"
server_key: "a596d937b5a18edc3e126176b3c2b66b"
server_key_iv: "c86791c9f0b4393fb2ba137694095b1a"
server_key_version: "encryptionkey@0"
I20260806 22:32:03.543888 30746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:32:03.547313 30763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:03.548708 30746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:03.548913 30746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "3470018af2c94d3091be7aa577691e45"
format_stamp: "Formatted at 2026-08-06 22:32:03 on dist-test-slave-7vjv"
server_key: "a596d937b5a18edc3e126176b3c2b66b"
server_key_iv: "c86791c9f0b4393fb2ba137694095b1a"
server_key_version: "encryptionkey@0"
I20260806 22:32:03.549150 30746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:03.593005 30746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:03.593878 30746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:03.594187 30746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:03.615366 30746 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38287
I20260806 22:32:03.615391 30838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38287 every 8 connection(s)
I20260806 22:32:03.616765 30746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:03.619751 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 30746
I20260806 22:32:03.619938 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:03.620537 1057 external_mini_cluster.cc:1428] Setting key 8fbcf31d9f8ba4f614384b5c99e89c41
I20260806 22:32:03.620911 30841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:03.631973 30841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: Bootstrap starting.
I20260806 22:32:03.635650 30841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:03.636973 30841 log.cc:824] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:03.640322 30841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: No bootstrap required, opened a new log
I20260806 22:32:03.640625 30845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:03.622833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.1.8.126:38287)
0806 22:32:03.624190 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:03.624206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:03.628753 (+ 4547us) server_negotiation.cc:407] Connection header received
0806 22:32:03.630744 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:03.630766 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:03.631243 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:03.631568 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:03.632265 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:03.633808 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:03.635874 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:03.636659 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:03.636797 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:03.638149 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:03.638164 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:03.638184 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:03.638821 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:03.638897 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:03.638919 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:03.639013 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:03.639247 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:32:03.639586 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":504,"threads_started":1}
I20260806 22:32:03.647904 30841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } }
I20260806 22:32:03.648373 30841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:03.648502 30841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3470018af2c94d3091be7aa577691e45, State: Initialized, Role: FOLLOWER
I20260806 22:32:03.649237 30841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } }
I20260806 22:32:03.649505 30841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:03.649650 30841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:03.649868 30841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:03.652211 30841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } }
I20260806 22:32:03.652727 30841 leader_election.cc:302] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3470018af2c94d3091be7aa577691e45; no voters:
I20260806 22:32:03.653268 30841 leader_election.cc:288] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:03.653429 30849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:03.654338 30849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [term 1 LEADER]: Becoming Leader. State: Replica: 3470018af2c94d3091be7aa577691e45, State: Running, Role: LEADER
I20260806 22:32:03.654855 30849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } }
I20260806 22:32:03.655622 30841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:03.657867 30850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3470018af2c94d3091be7aa577691e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } } }
I20260806 22:32:03.657867 30851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3470018af2c94d3091be7aa577691e45. Latest consensus state: current_term: 1 leader_uuid: "3470018af2c94d3091be7aa577691e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3470018af2c94d3091be7aa577691e45" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38287 } } }
I20260806 22:32:03.658330 30850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:03.658334 30851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:03.660056 30854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:03.664752 30854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:03.673825 30854 catalog_manager.cc:1347] Generated new cluster ID: 1820208e633b48f9a159aafaeb12bd98
I20260806 22:32:03.673940 30854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:03.694115 30854 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:03.695389 30854 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:03.706538 30854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: Generated new TSK 0
I20260806 22:32:03.707317 30854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:03.721781 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--builtin_ntp_servers=127.1.8.84:34643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:32:04.073791 30878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:04.074059 30878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:04.074148 30878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:04.081351 30878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:04.081452 30878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:04.081635 30878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:32:04.090627 30878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:04.091959 30878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:04.093827 30878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:04.106963 30888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:04.107007 30884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:04.107052 30885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:04.107786 30878 server_base.cc:1034] running on GCE node
I20260806 22:32:04.108438 30878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:04.109335 30878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:04.110546 30878 hybrid_clock.cc:648] HybridClock initialized: now 1786055524110504 us; error 28 us; skew 500 ppm
I20260806 22:32:04.110812 30878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:04.113024 30878 webserver.cc:460] Webserver started at http://127.1.8.65:46231/ using document root <none> and password file <none>
I20260806 22:32:04.113775 30878 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:04.113865 30878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:04.114136 30878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:04.116940 30878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "a7304f99f8da4506a0a05079d57976ce"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "5fce92eb12b27b713e7c4bee0765521e"
server_key_iv: "a599dff068d33095dd26639415c4630e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.117640 30878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "a7304f99f8da4506a0a05079d57976ce"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "5fce92eb12b27b713e7c4bee0765521e"
server_key_iv: "a599dff068d33095dd26639415c4630e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.122466 30878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:04.125715 30897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.126896 30878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:32:04.127064 30878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "a7304f99f8da4506a0a05079d57976ce"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "5fce92eb12b27b713e7c4bee0765521e"
server_key_iv: "a599dff068d33095dd26639415c4630e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.127272 30878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:04.164826 30878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:04.165733 30878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:04.166015 30878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:04.166968 30878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:04.168476 30878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:04.168615 30878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.168738 30878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:04.168826 30878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.201715 30878 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:39711
I20260806 22:32:04.201771 31068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:39711 every 8 connection(s)
I20260806 22:32:04.203023 30878 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:32:04.205351 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 30878
I20260806 22:32:04.205538 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:32:04.206164 1057 external_mini_cluster.cc:1428] Setting key 75e4b8c13898515b145661c42d4f7834
I20260806 22:32:04.209422 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--builtin_ntp_servers=127.1.8.84:34643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:32:04.217890 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:04.205772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:37319 (local address 127.1.8.126:38287)
0806 22:32:04.206271 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:04.206279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:04.207554 (+ 1275us) server_negotiation.cc:407] Connection header received
0806 22:32:04.210145 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:04.210149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:04.210220 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:04.210305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:04.211769 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.212515 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.215003 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.215406 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.215499 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:04.217134 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:04.217139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:04.217142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:04.217160 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:04.217198 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:04.217210 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:04.217305 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:04.217546 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 22:32:04.217622 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260806 22:32:04.219353 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:04.206333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38287 (local address 127.1.8.65:37319)
0806 22:32:04.207418 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:04.207453 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:04.209841 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:04.210590 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:04.210604 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:04.210731 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:04.211598 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:04.211626 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.212716 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.212724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:04.214859 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:04.214878 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.215655 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.215665 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:04.216000 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:04.216759 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:04.216765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:04.216831 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:04.216936 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:04.217331 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:04.217343 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:04.217354 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:32:04.217532 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 22:32:04.217978 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":350,"threads_started":1}
I20260806 22:32:04.220970 31069 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38287
I20260806 22:32:04.221292 31069 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:04.222206 31069 heartbeater.cc:511] Master 127.1.8.126:38287 requested a full tablet report, sending...
I20260806 22:32:04.224414 30789 ts_manager.cc:194] Registered new tserver with Master: a7304f99f8da4506a0a05079d57976ce (127.1.8.65:39711)
I20260806 22:32:04.226307 30789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.65:37319
W20260806 22:32:04.560174 31074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:04.560449 31074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:04.560623 31074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:04.567898 31074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:04.567998 31074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:04.568229 31074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:32:04.577224 31074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:04.578557 31074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:04.580374 31074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:04.593694 31082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:04.593765 31084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:04.593737 31088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:04.594269 31074 server_base.cc:1034] running on GCE node
I20260806 22:32:04.594929 31074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:04.595868 31074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:04.597114 31074 hybrid_clock.cc:648] HybridClock initialized: now 1786055524597052 us; error 71 us; skew 500 ppm
I20260806 22:32:04.597434 31074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:04.599715 31074 webserver.cc:460] Webserver started at http://127.1.8.66:39507/ using document root <none> and password file <none>
I20260806 22:32:04.600476 31074 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:04.600612 31074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:04.600940 31074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:04.603766 31074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "c150185320f343269f477bd3ad1aa62a"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "771d9d461b659eeb003afad5a08aaad0"
server_key_iv: "dc1e7683d5b19876d4fe980494bf270e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.604589 31074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "c150185320f343269f477bd3ad1aa62a"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "771d9d461b659eeb003afad5a08aaad0"
server_key_iv: "dc1e7683d5b19876d4fe980494bf270e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.609701 31074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:04.613181 31094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.614477 31074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:32:04.614655 31074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "c150185320f343269f477bd3ad1aa62a"
format_stamp: "Formatted at 2026-08-06 22:32:04 on dist-test-slave-7vjv"
server_key: "771d9d461b659eeb003afad5a08aaad0"
server_key_iv: "dc1e7683d5b19876d4fe980494bf270e"
server_key_version: "encryptionkey@0"
I20260806 22:32:04.614898 31074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:04.632627 31074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:04.633581 31074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:04.633872 31074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:04.634969 31074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:04.636770 31074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:04.636878 31074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.636981 31074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:04.637051 31074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:04.671872 31074 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:35411
I20260806 22:32:04.671949 31272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:35411 every 8 connection(s)
I20260806 22:32:04.673260 31074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:32:04.677790 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 31074
I20260806 22:32:04.677981 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:32:04.678481 1057 external_mini_cluster.cc:1428] Setting key 5d37b76c314fb4c12a10d0ff8aa080fa
I20260806 22:32:04.681512 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--builtin_ntp_servers=127.1.8.84:34643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:32:04.687045 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:04.675862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:60901 (local address 127.1.8.126:38287)
0806 22:32:04.676043 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:04.676047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:04.677370 (+ 1323us) server_negotiation.cc:407] Connection header received
0806 22:32:04.679429 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:04.679435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:04.679554 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:04.679676 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:04.680962 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.681728 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.684256 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.684658 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.684752 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:04.686360 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:04.686365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:04.686368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:04.686387 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:04.686418 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:04.686430 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:04.686494 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:04.686749 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 22:32:04.686826 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:32:04.688593 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:04.676337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38287 (local address 127.1.8.66:60901)
0806 22:32:04.677208 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:04.677245 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:04.679116 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:04.679879 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:04.679887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:04.679967 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:04.680770 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:04.680796 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.681876 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.681884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:04.684083 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:04.684134 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:04.684885 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:04.684895 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:04.685230 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:04.685987 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:04.685992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:04.686048 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:04.686158 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:04.686523 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:04.686536 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:04.686547 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:32:04.686721 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 22:32:04.687147 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":262,"threads_started":1}
I20260806 22:32:04.690074 31273 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38287
I20260806 22:32:04.690385 31273 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:04.691123 31273 heartbeater.cc:511] Master 127.1.8.126:38287 requested a full tablet report, sending...
I20260806 22:32:04.692621 30789 ts_manager.cc:194] Registered new tserver with Master: c150185320f343269f477bd3ad1aa62a (127.1.8.66:35411)
I20260806 22:32:04.693428 30789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.66:60901
W20260806 22:32:05.034711 31279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:05.035017 31279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:05.035183 31279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:05.042063 31279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:05.042150 31279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:05.042250 31279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:32:05.050781 31279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:38287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:05.051956 31279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:05.053893 31279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:05.067555 31291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:05.067515 31293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:05.067625 31289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:05.068504 31279 server_base.cc:1034] running on GCE node
I20260806 22:32:05.069123 31279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:05.070088 31279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:05.071295 31279 hybrid_clock.cc:648] HybridClock initialized: now 1786055525071243 us; error 45 us; skew 500 ppm
I20260806 22:32:05.071568 31279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:05.073936 31279 webserver.cc:460] Webserver started at http://127.1.8.67:33559/ using document root <none> and password file <none>
I20260806 22:32:05.074605 31279 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:05.074695 31279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:05.074970 31279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:05.077984 31279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "cded455b7404488abd2322238674fac2"
format_stamp: "Formatted at 2026-08-06 22:32:05 on dist-test-slave-7vjv"
server_key: "a00458fff10cd31b784a8fe7d147e865"
server_key_iv: "e9fca09853ed440b4a42dfd921947949"
server_key_version: "encryptionkey@0"
I20260806 22:32:05.078728 31279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "cded455b7404488abd2322238674fac2"
format_stamp: "Formatted at 2026-08-06 22:32:05 on dist-test-slave-7vjv"
server_key: "a00458fff10cd31b784a8fe7d147e865"
server_key_iv: "e9fca09853ed440b4a42dfd921947949"
server_key_version: "encryptionkey@0"
I20260806 22:32:05.083760 31279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 22:32:05.087198 31302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:05.088526 31279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:05.088714 31279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "cded455b7404488abd2322238674fac2"
format_stamp: "Formatted at 2026-08-06 22:32:05 on dist-test-slave-7vjv"
server_key: "a00458fff10cd31b784a8fe7d147e865"
server_key_iv: "e9fca09853ed440b4a42dfd921947949"
server_key_version: "encryptionkey@0"
I20260806 22:32:05.088924 31279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:05.105345 31279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:05.106164 31279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:05.106483 31279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:05.107406 31279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:05.108932 31279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:05.109054 31279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:05.109177 31279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:05.109272 31279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:05.143167 31279 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:42201
I20260806 22:32:05.143229 31474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:42201 every 8 connection(s)
I20260806 22:32:05.144621 31279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:32:05.147246 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 31279
I20260806 22:32:05.147451 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:32:05.147926 1057 external_mini_cluster.cc:1428] Setting key 8a2e72d5db26f9315260a5cdfb6dc24f
I20260806 22:32:05.157395 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.147327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:37941 (local address 127.1.8.126:38287)
0806 22:32:05.147590 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.147598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.149106 (+ 1508us) server_negotiation.cc:407] Connection header received
0806 22:32:05.151054 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.151058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.151127 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.151210 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:05.152294 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.153098 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.154790 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.155189 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.155283 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.156640 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:05.156645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:05.156648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:05.156688 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:05.156741 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:05.156755 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:05.156828 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.157030 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.157107 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 22:32:05.158444 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.147886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:38287 (local address 127.1.8.67:37941)
0806 22:32:05.148949 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.148985 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.150765 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.151417 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.151424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.151508 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:05.152077 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.152146 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.153237 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.153241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.154661 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.154674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.155376 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.155381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.155577 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.156321 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:05.156324 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:05.156363 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:05.156442 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:05.156839 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:05.156847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:05.156853 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.157022 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.157386 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":383,"threads_started":1}
I20260806 22:32:05.159807 31475 heartbeater.cc:347] Connected to a master server at 127.1.8.126:38287
I20260806 22:32:05.160166 31475 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:05.160961 31475 heartbeater.cc:511] Master 127.1.8.126:38287 requested a full tablet report, sending...
I20260806 22:32:05.162355 30789 ts_manager.cc:194] Registered new tserver with Master: cded455b7404488abd2322238674fac2 (127.1.8.67:42201)
I20260806 22:32:05.163228 30789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.67:37941
I20260806 22:32:05.163460 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:32:05.173416 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.167846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.1.8.126:38287)
0806 22:32:05.168005 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.168009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.168517 (+ 508us) server_negotiation.cc:407] Connection header received
0806 22:32:05.168658 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.168662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.168733 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.168814 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:05.169241 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.169897 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.171247 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.171666 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.171755 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.172822 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:05.172827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:05.172830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:05.172846 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:05.172876 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:05.172887 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:05.172996 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.173151 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.173196 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:32:05.177208 30789 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:32:05.183987 30789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:32:05.212528 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.196609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:42201 (local address 127.0.0.1:47392)
0806 22:32:05.200252 (+ 3643us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.200277 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.200422 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.201644 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.201656 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.201697 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:05.202244 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.202261 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.204954 (+ 2693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.204963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.208112 (+ 3149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.208136 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.211186 (+ 3050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.211196 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.211503 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.211589 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.211748 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.212013 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3438,"threads_started":1}
I20260806 22:32:05.213515 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.197383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47392 (local address 127.1.8.67:42201)
0806 22:32:05.199743 (+ 2360us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.199755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.200576 (+ 821us) server_negotiation.cc:407] Connection header received
0806 22:32:05.200723 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.200735 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.201050 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.201258 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:05.202541 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.204727 (+ 2186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.208424 (+ 3697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.210430 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.210647 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.211007 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.212794 (+ 1787us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.213001 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2125,"thread_start_us":1949,"threads_started":1}
I20260806 22:32:05.217645 31375 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa78911412450f8dedc68f57c56e6a (DEFAULT_TABLE table=test-table [id=ad7770d624294e9f8be971c42ae04ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:05.219386 31375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa78911412450f8dedc68f57c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:05.223237 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.196375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:39711 (local address 127.0.0.1:54812)
0806 22:32:05.197483 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.197527 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.197737 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.214885 (+ 17148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.214890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.214915 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:05.215313 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.215329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.217567 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.217580 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.220048 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.220064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.222197 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.222205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.222419 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.222463 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.222584 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.222737 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":120,"thread_start_us":1548,"threads_started":1}
I20260806 22:32:05.223250 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.196545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.1.8.65:39711)
0806 22:32:05.214206 (+ 17661us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.214213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.214238 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:32:05.214313 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.214352 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.214648 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.215521 (+ 873us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:05.215791 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.217405 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.220252 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.221753 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.221892 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.222186 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.222660 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.222854 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17519,"thread_start_us":17393,"threads_started":1}
I20260806 22:32:05.223968 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.197851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38842 (local address 127.1.8.66:35411)
0806 22:32:05.214525 (+ 16674us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.214537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.214579 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:32:05.214686 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.214693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.214908 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.215116 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:05.216024 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.217555 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.219786 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.222026 (+ 2240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.222181 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.222401 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.223591 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.223711 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16496,"thread_start_us":15841,"threads_started":1}
I20260806 22:32:05.224200 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.197261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:35411 (local address 127.0.0.1:38842)
0806 22:32:05.212640 (+ 15379us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.212667 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.212821 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.215387 (+ 2566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.215392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.215418 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:05.215816 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.215829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.217709 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.217716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.219587 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.219614 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.222514 (+ 2900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.222519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.222710 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.222741 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.222829 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.222954 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15199,"mutex_wait_us":128,"thread_start_us":2648,"threads_started":1}
I20260806 22:32:05.227030 30963 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa78911412450f8dedc68f57c56e6a (DEFAULT_TABLE table=test-table [id=ad7770d624294e9f8be971c42ae04ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:05.228394 30963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa78911412450f8dedc68f57c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:05.227965 31172 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa78911412450f8dedc68f57c56e6a (DEFAULT_TABLE table=test-table [id=ad7770d624294e9f8be971c42ae04ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:05.229180 31069 heartbeater.cc:503] Master 127.1.8.126:38287 was elected leader, sending a full tablet report...
I20260806 22:32:05.229472 31172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa78911412450f8dedc68f57c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:05.235566 31503 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Bootstrap starting.
I20260806 22:32:05.239308 31503 tablet_bootstrap.cc:654] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:05.240653 31503 log.cc:824] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:05.243623 31507 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Bootstrap starting.
I20260806 22:32:05.244050 31503 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: No bootstrap required, opened a new log
I20260806 22:32:05.244385 31503 ts_tablet_manager.cc:1397] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260806 22:32:05.248765 31508 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Bootstrap starting.
I20260806 22:32:05.249985 31507 tablet_bootstrap.cc:654] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:05.251107 31507 log.cc:824] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:05.252548 31508 tablet_bootstrap.cc:654] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:05.253773 31508 log.cc:824] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:05.253497 31503 raft_consensus.cc:348] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.253963 31503 raft_consensus.cc:374] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:05.254127 31503 raft_consensus.cc:729] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cded455b7404488abd2322238674fac2, State: Initialized, Role: FOLLOWER
I20260806 22:32:05.254392 31507 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: No bootstrap required, opened a new log
I20260806 22:32:05.254779 31507 ts_tablet_manager.cc:1397] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 22:32:05.254948 31503 consensus_queue.cc:260] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.256289 31475 heartbeater.cc:503] Master 127.1.8.126:38287 was elected leader, sending a full tablet report...
I20260806 22:32:05.258967 31503 ts_tablet_manager.cc:1428] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260806 22:32:05.261982 31508 tablet_bootstrap.cc:492] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: No bootstrap required, opened a new log
I20260806 22:32:05.262283 31508 ts_tablet_manager.cc:1397] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260806 22:32:05.266861 31507 raft_consensus.cc:348] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.267325 31507 raft_consensus.cc:374] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:05.267436 31507 raft_consensus.cc:729] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7304f99f8da4506a0a05079d57976ce, State: Initialized, Role: FOLLOWER
I20260806 22:32:05.268110 31507 consensus_queue.cc:260] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.269901 31507 ts_tablet_manager.cc:1428] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260806 22:32:05.272421 31508 raft_consensus.cc:348] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.272931 31508 raft_consensus.cc:374] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:05.273046 31508 raft_consensus.cc:729] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c150185320f343269f477bd3ad1aa62a, State: Initialized, Role: FOLLOWER
I20260806 22:32:05.273703 31508 consensus_queue.cc:260] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.274689 31273 heartbeater.cc:503] Master 127.1.8.126:38287 was elected leader, sending a full tablet report...
I20260806 22:32:05.275179 31508 ts_tablet_manager.cc:1428] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 22:32:05.385557 31514 raft_consensus.cc:482] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:32:05.385869 31514 raft_consensus.cc:504] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.387639 31514 leader_election.cc:288] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7304f99f8da4506a0a05079d57976ce (127.1.8.65:39711), c150185320f343269f477bd3ad1aa62a (127.1.8.66:35411)
I20260806 22:32:05.394379 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.388038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:37779 (local address 127.1.8.65:39711)
0806 22:32:05.388290 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.388299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.388327 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:32:05.388523 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.388532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.388650 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.388757 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:05.389262 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.390318 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.392504 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.393596 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.393688 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.393759 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.394067 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.394175 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260806 22:32:05.394524 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.387863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:39711 (local address 127.1.8.67:37779)
0806 22:32:05.388177 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.388201 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.388337 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.388804 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.388810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.388840 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:05.389085 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.389096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.390460 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.390465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.392347 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.392361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.393732 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.393738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.393903 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.393937 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.394038 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.394193 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":36}
I20260806 22:32:05.395375 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.388179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:58433 (local address 127.1.8.66:35411)
0806 22:32:05.388370 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.388377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.389052 (+ 675us) server_negotiation.cc:407] Connection header received
0806 22:32:05.389272 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.389277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.389369 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.389432 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:05.390156 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.391185 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.393298 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.394372 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.394528 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.394635 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.394984 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.395120 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:32:05.395428 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.388085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:35411 (local address 127.1.8.67:58433)
0806 22:32:05.388938 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:05.388958 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:05.389089 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:05.389616 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:05.389620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:05.389644 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:05.389970 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.389980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.391334 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.391338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.393064 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:05.393091 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.394669 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.394675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:05.394828 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.394851 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:32:05.394958 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:32:05.395076 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":2,"thread_start_us":459,"threads_started":1}
I20260806 22:32:05.395640 30999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0fa78911412450f8dedc68f57c56e6a" candidate_uuid: "cded455b7404488abd2322238674fac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7304f99f8da4506a0a05079d57976ce" is_pre_election: true
W20260806 22:32:05.396334 31477 tablet.cc:2369] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:05.396303 30999 raft_consensus.cc:2454] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cded455b7404488abd2322238674fac2 in term 0.
I20260806 22:32:05.396498 31196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0fa78911412450f8dedc68f57c56e6a" candidate_uuid: "cded455b7404488abd2322238674fac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c150185320f343269f477bd3ad1aa62a" is_pre_election: true
I20260806 22:32:05.397037 31196 raft_consensus.cc:2454] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cded455b7404488abd2322238674fac2 in term 0.
I20260806 22:32:05.397138 31304 leader_election.cc:302] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7304f99f8da4506a0a05079d57976ce, cded455b7404488abd2322238674fac2; no voters:
I20260806 22:32:05.397562 31514 raft_consensus.cc:2781] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:32:05.397742 31514 raft_consensus.cc:482] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:32:05.397872 31514 raft_consensus.cc:3037] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:05.400074 31514 raft_consensus.cc:504] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.400717 31514 leader_election.cc:288] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [CANDIDATE]: Term 1 election: Requested vote from peers a7304f99f8da4506a0a05079d57976ce (127.1.8.65:39711), c150185320f343269f477bd3ad1aa62a (127.1.8.66:35411)
I20260806 22:32:05.401191 31196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0fa78911412450f8dedc68f57c56e6a" candidate_uuid: "cded455b7404488abd2322238674fac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c150185320f343269f477bd3ad1aa62a"
I20260806 22:32:05.401232 30999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0fa78911412450f8dedc68f57c56e6a" candidate_uuid: "cded455b7404488abd2322238674fac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7304f99f8da4506a0a05079d57976ce"
I20260806 22:32:05.401382 31196 raft_consensus.cc:3037] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:05.401420 30999 raft_consensus.cc:3037] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:05.403652 30999 raft_consensus.cc:2454] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cded455b7404488abd2322238674fac2 in term 1.
I20260806 22:32:05.403717 31196 raft_consensus.cc:2454] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cded455b7404488abd2322238674fac2 in term 1.
I20260806 22:32:05.404220 31308 leader_election.cc:302] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c150185320f343269f477bd3ad1aa62a, cded455b7404488abd2322238674fac2; no voters:
I20260806 22:32:05.404502 31514 raft_consensus.cc:2781] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:05.404919 31514 raft_consensus.cc:686] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 1 LEADER]: Becoming Leader. State: Replica: cded455b7404488abd2322238674fac2, State: Running, Role: LEADER
I20260806 22:32:05.405391 31514 consensus_queue.cc:237] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } }
I20260806 22:32:05.410063 30789 catalog_manager.cc:5495] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cded455b7404488abd2322238674fac2 (127.1.8.67). New cstate: current_term: 1 leader_uuid: "cded455b7404488abd2322238674fac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cded455b7404488abd2322238674fac2" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 } health_report { overall_health: UNKNOWN } } }
W20260806 22:32:05.425308 31276 tablet.cc:2369] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:05.448935 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.443564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.1.8.67:42201)
0806 22:32:05.443793 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.443798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.443817 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:32:05.444008 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.444012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.444169 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.444304 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:32:05.444656 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.445387 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.446727 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.447133 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.447223 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.447700 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:32:05.448304 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:32:05.448443 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.448578 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.448642 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 22:32:05.455611 31070 tablet.cc:2369] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:05.457362 31196 raft_consensus.cc:1261] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 1 FOLLOWER]: Refusing update from remote peer cded455b7404488abd2322238674fac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:05.457365 30999 raft_consensus.cc:1261] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 1 FOLLOWER]: Refusing update from remote peer cded455b7404488abd2322238674fac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:05.458326 31514 consensus_queue.cc:1035] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7304f99f8da4506a0a05079d57976ce" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 39711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:05.458657 31519 consensus_queue.cc:1035] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c150185320f343269f477bd3ad1aa62a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:05.468416 31524 mvcc.cc:205] Tried to move back new op lower bound from 7315683432262479872 to 7315683432063954944. Current Snapshot: MvccSnapshot[applied={T|T < 7315683432262479872}]
I20260806 22:32:05.486934 30789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:05.487166 30789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:05.491039 30789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3470018af2c94d3091be7aa577691e45: Sending DeleteTablet for 3 replicas of tablet c0fa78911412450f8dedc68f57c56e6a
I20260806 22:32:05.492311 31375 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa78911412450f8dedc68f57c56e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:05 UTC) from {username='slave'} at 127.0.0.1:47392
I20260806 22:32:05.492317 31172 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa78911412450f8dedc68f57c56e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:05 UTC) from {username='slave'} at 127.0.0.1:38842
I20260806 22:32:05.492306 30963 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa78911412450f8dedc68f57c56e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:05 UTC) from {username='slave'} at 127.0.0.1:54812
I20260806 22:32:05.495033 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 30878
I20260806 22:32:05.503537 31539 tablet_replica.cc:330] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: stopping tablet replica
I20260806 22:32:05.504210 31539 raft_consensus.cc:2229] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:05.504675 31541 tablet_replica.cc:330] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: stopping tablet replica
I20260806 22:32:05.504944 31539 raft_consensus.cc:2258] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:05.505230 31541 raft_consensus.cc:2229] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:05.505755 31541 raft_consensus.cc:2258] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:05.506654 31537 tablet_replica.cc:330] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: stopping tablet replica
I20260806 22:32:05.508060 31539 ts_tablet_manager.cc:1905] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:05.508592 31537 raft_consensus.cc:2229] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:32:05.509128 31541 ts_tablet_manager.cc:1905] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:05.509855 31537 raft_consensus.cc:2258] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:05.516131 31539 ts_tablet_manager.cc:1918] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:05.516420 31539 log.cc:1196] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/c0fa78911412450f8dedc68f57c56e6a
I20260806 22:32:05.517122 31539 ts_tablet_manager.cc:1939] T c0fa78911412450f8dedc68f57c56e6a P c150185320f343269f477bd3ad1aa62a: Deleting consensus metadata
I20260806 22:32:05.517648 31541 ts_tablet_manager.cc:1918] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:05.517894 31541 log.cc:1196] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/c0fa78911412450f8dedc68f57c56e6a
I20260806 22:32:05.518519 31541 ts_tablet_manager.cc:1939] T c0fa78911412450f8dedc68f57c56e6a P a7304f99f8da4506a0a05079d57976ce: Deleting consensus metadata
I20260806 22:32:05.518507 31537 ts_tablet_manager.cc:1905] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:05.519243 30767 catalog_manager.cc:4841] TS c150185320f343269f477bd3ad1aa62a (127.1.8.66:35411): tablet c0fa78911412450f8dedc68f57c56e6a (table test-table [id=ad7770d624294e9f8be971c42ae04ef6]) successfully deleted
I20260806 22:32:05.520551 30764 catalog_manager.cc:4841] TS a7304f99f8da4506a0a05079d57976ce (127.1.8.65:39711): tablet c0fa78911412450f8dedc68f57c56e6a (table test-table [id=ad7770d624294e9f8be971c42ae04ef6]) successfully deleted
I20260806 22:32:05.524700 31537 ts_tablet_manager.cc:1918] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:05.524897 31537 log.cc:1196] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/c0fa78911412450f8dedc68f57c56e6a
I20260806 22:32:05.525538 31537 ts_tablet_manager.cc:1939] T c0fa78911412450f8dedc68f57c56e6a P cded455b7404488abd2322238674fac2: Deleting consensus metadata
I20260806 22:32:05.527292 30765 catalog_manager.cc:4841] TS cded455b7404488abd2322238674fac2 (127.1.8.67:42201): tablet c0fa78911412450f8dedc68f57c56e6a (table test-table [id=ad7770d624294e9f8be971c42ae04ef6]) successfully deleted
I20260806 22:32:05.528473 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.496710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.1.8.65:39711)
0806 22:32:05.496945 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.496955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.511663 (+ 14708us) server_negotiation.cc:407] Connection header received
0806 22:32:05.512160 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.512166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.512270 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.512363 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:05.513117 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.514251 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.520593 (+ 6342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.521269 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.521376 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.526728 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:05.526741 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:05.526758 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:05.527553 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:05.527625 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:05.527646 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:05.527748 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.527917 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.528016 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 22:32:05.529273 31062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:05.621652 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 30878
I20260806 22:32:05.650394 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 31074
I20260806 22:32:05.658927 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.651232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.1.8.66:35411)
0806 22:32:05.651391 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.651396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.651415 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:32:05.651614 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.651618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.651705 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.651790 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:05.652315 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.653166 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.655021 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.655416 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.655514 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.657503 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:05.657512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:05.657527 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:05.658144 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:05.658196 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:05.658216 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:05.658306 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.658471 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.658561 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:32:05.659682 31267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:05.753299 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 31074
I20260806 22:32:05.790779 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 31279
I20260806 22:32:05.798996 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:05.791599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.1.8.67:42201)
0806 22:32:05.792133 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:05.792141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:05.792181 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:05.792237 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:05.792240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:05.792352 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:05.792478 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:05.792978 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.793838 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.795795 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:05.796268 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:05.796362 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:05.797535 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:05.797543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:05.797558 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:05.798189 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:05.798237 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:05.798257 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:05.798334 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:05.798525 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:32:05.798613 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360}
I20260806 22:32:05.799750 31469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:05.923164 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 31279
I20260806 22:32:05.950314 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 30746
I20260806 22:32:05.951439 30829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:06.063457 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 30746
2026-08-06T22:32:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3011 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:06 dist-test-slave-7vjv krb5kdc[31552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:06 dist-test-slave-7vjv krb5kdc[31552](info): set up 2 sockets
Aug 06 22:32:06 dist-test-slave-7vjv krb5kdc[31552](info): commencing operation
krb5kdc: starting...
W20260806 22:32:07.406395 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:07 dist-test-slave-7vjv krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:07.729686 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43869
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:43869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:08.080274 31568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:08.080546 31568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:08.080741 31568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:08.087898 31568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:08.087999 31568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:08.088086 31568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:08.088214 31568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:08.088279 31568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:08.097949 31568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:43869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:08.099213 31568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:08.100641 31568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:08.112902 31573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:08.112947 31574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:08.113080 31576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:08.113952 31568 server_base.cc:1034] running on GCE node
I20260806 22:32:08.114693 31568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:08.116086 31568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:08.117341 31568 hybrid_clock.cc:648] HybridClock initialized: now 1786055528117290 us; error 54 us; skew 500 ppm
I20260806 22:32:08.117719 31568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:08 dist-test-slave-7vjv krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:08.120987 31568 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:08.122532 31568 webserver.cc:460] Webserver started at http://127.1.8.126:44643/ using document root <none> and password file <none>
I20260806 22:32:08.123220 31568 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:08.123354 31568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:08.123678 31568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:08.126703 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "22d9793463154e50bd50cb2254174961"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "70533667632cdad84f77e30eaf6a5949"
server_key_iv: "bd4d1b83752995400ce58b2f35a2d861"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.127532 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "22d9793463154e50bd50cb2254174961"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "70533667632cdad84f77e30eaf6a5949"
server_key_iv: "bd4d1b83752995400ce58b2f35a2d861"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.132761 31568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 22:32:08.136049 31589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:08.137315 31568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:08.137502 31568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "22d9793463154e50bd50cb2254174961"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "70533667632cdad84f77e30eaf6a5949"
server_key_iv: "bd4d1b83752995400ce58b2f35a2d861"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.137718 31568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:08.150272 31568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:08.154521 31568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:08.154805 31568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:08.173228 31568 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:43869
I20260806 22:32:08.173308 31667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:43869 every 8 connection(s)
I20260806 22:32:08.174588 31568 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:08.178185 31671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:08.183681 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 31568
I20260806 22:32:08.183884 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:08.184407 1057 external_mini_cluster.cc:1428] Setting key 5a791c4d4906f0f2655dc92485407363
I20260806 22:32:08.189040 31671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: Bootstrap starting.
I20260806 22:32:08.192732 31671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:08.194012 31671 log.cc:824] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:08.197371 31671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: No bootstrap required, opened a new log
Aug 06 22:32:08 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:08.207777 31671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } }
I20260806 22:32:08.208236 31671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:08.208352 31671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d9793463154e50bd50cb2254174961, State: Initialized, Role: FOLLOWER
I20260806 22:32:08.209019 31671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } }
I20260806 22:32:08.209263 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:08.186203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.1.8.126:43869)
0806 22:32:08.187037 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:08.187052 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:08.187108 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:32:08.188828 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:08.188849 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:08.189259 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:08.189580 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:08.191514 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.192813 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.194745 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.195677 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.195802 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:08.200735 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:08.200745 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:08.200775 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:08.200823 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:08.204850 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:08.205511 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:08.205520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:08.205527 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:08.205760 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:08.206147 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:08.206153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:08.206156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:08.206838 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:08.207135 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:08.207442 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 22:32:08.207859 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":413,"threads_started":1}
I20260806 22:32:08.209334 31671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:08.209468 31671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:08.209666 31671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:08.211934 31671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } }
I20260806 22:32:08.212433 31671 leader_election.cc:302] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22d9793463154e50bd50cb2254174961; no voters:
I20260806 22:32:08.212991 31671 leader_election.cc:288] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:08.213135 31677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:08.213819 31677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [term 1 LEADER]: Becoming Leader. State: Replica: 22d9793463154e50bd50cb2254174961, State: Running, Role: LEADER
I20260806 22:32:08.214324 31677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } }
I20260806 22:32:08.214936 31671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:08.217389 31679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22d9793463154e50bd50cb2254174961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } } }
I20260806 22:32:08.217633 31681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22d9793463154e50bd50cb2254174961. Latest consensus state: current_term: 1 leader_uuid: "22d9793463154e50bd50cb2254174961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d9793463154e50bd50cb2254174961" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43869 } } }
I20260806 22:32:08.217886 31679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:08.217931 31681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:08.218815 31687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:08.224803 31687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:08.234110 31687 catalog_manager.cc:1347] Generated new cluster ID: e1066e392e7440a7a42dc103cdcbfc1d
I20260806 22:32:08.234228 31687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:08.255452 31687 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:08.256918 31687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:08.263857 31687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: Generated new TSK 0
I20260806 22:32:08.264703 31687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
Entry for principal kudu/127.1.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab.
I20260806 22:32:08.353191 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:08.719034 31715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:08.719295 31715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:08.719388 31715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:08.726752 31715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:08.726861 31715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:08.726989 31715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:32:08.736505 31715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.65.keytab
--principal=kudu/127.1.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:08.737668 31715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:08.738896 31715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:08.752055 31721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:08.752039 31722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:08.752198 31725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:08.753127 31715 server_base.cc:1034] running on GCE node
I20260806 22:32:08.753654 31715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:08.754518 31715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:08.755756 31715 hybrid_clock.cc:648] HybridClock initialized: now 1786055528755719 us; error 49 us; skew 500 ppm
I20260806 22:32:08.756078 31715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:08 dist-test-slave-7vjv krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:08.759382 31715 init.cc:377] Logged in from keytab as kudu/127.1.8.65@KRBTEST.COM (short username kudu)
I20260806 22:32:08.760843 31715 webserver.cc:460] Webserver started at http://127.1.8.65:34943/ using document root <none> and password file <none>
I20260806 22:32:08.761473 31715 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:08.761570 31715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:08.761821 31715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:08.764930 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "c8efff18118145db84f8810cfec35d93"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "82c41421b370c0712b799289ebbf26af"
server_key_iv: "e981ecb3359dfbf946df937047b9792b"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.765672 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "c8efff18118145db84f8810cfec35d93"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "82c41421b370c0712b799289ebbf26af"
server_key_iv: "e981ecb3359dfbf946df937047b9792b"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.770753 31715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:32:08.774087 31734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:08.775338 31715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:32:08.775470 31715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "c8efff18118145db84f8810cfec35d93"
format_stamp: "Formatted at 2026-08-06 22:32:08 on dist-test-slave-7vjv"
server_key: "82c41421b370c0712b799289ebbf26af"
server_key_iv: "e981ecb3359dfbf946df937047b9792b"
server_key_version: "encryptionkey@0"
I20260806 22:32:08.775626 31715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:08.793017 31715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:08.797233 31715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:08.797552 31715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:08.798425 31715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:08.799695 31715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:08.799767 31715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:08.799852 31715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:08.799897 31715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:08.832696 31715 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:38481
I20260806 22:32:08.832783 31914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:38481 every 8 connection(s)
I20260806 22:32:08.833973 31715 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:32:08.837234 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 31715
I20260806 22:32:08.837440 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:32:08.838025 1057 external_mini_cluster.cc:1428] Setting key a8ee3e0b995aea5b0153b8a3c1950c85
Aug 06 22:32:08 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.65@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:08.855648 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:08.836592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:35265 (local address 127.1.8.126:43869)
0806 22:32:08.837059 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:08.837063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:08.838071 (+ 1008us) server_negotiation.cc:407] Connection header received
0806 22:32:08.839990 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:08.839995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:08.840144 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:08.840264 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:08.841909 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.842720 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.844274 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.844701 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.844840 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:08.848744 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:08.848753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:08.848761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:08.848807 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:08.852490 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:08.853257 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:08.853265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:08.853270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:08.853428 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:08.853950 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:08.853956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:08.853962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:08.854329 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:08.854489 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:08.855148 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 22:32:08.855379 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":257,"threads_started":1}
I20260806 22:32:08.857158 31920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:08.837116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43869 (local address 127.1.8.65:35265)
0806 22:32:08.837909 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:08.837955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:08.839735 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:08.840502 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:08.840510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:08.841145 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:08.841747 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:08.841767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.842884 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.842897 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:08.844076 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:08.844142 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:08.844961 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:08.844971 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:08.845215 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:08.845904 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:08.845931 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:08.848475 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:08.852649 (+ 4174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:08.852657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:08.852671 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:08.853131 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:08.853644 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:08.853647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:08.853649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:08.853838 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:08.854449 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:08.854454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:08.854760 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 22:32:08.855010 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 22:32:08.855603 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":354,"threads_started":1}
I20260806 22:32:08.858778 31915 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43869
I20260806 22:32:08.859084 31915 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:08.859845 31915 heartbeater.cc:511] Master 127.1.8.126:43869 requested a full tablet report, sending...
I20260806 22:32:08.861961 31613 ts_manager.cc:194] Registered new tserver with Master: c8efff18118145db84f8810cfec35d93 (127.1.8.65:38481)
I20260806 22:32:08.863392 31613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.65@KRBTEST.COM'} at 127.1.8.65:35265
WARNING: no policy specified for kudu/127.1.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
Entry for principal kudu/127.1.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab.
I20260806 22:32:08.910843 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:09.251114 31924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:09.251430 31924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:09.251614 31924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:09.258981 31924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:09.259141 31924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:09.259317 31924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:32:09.269006 31924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.66.keytab
--principal=kudu/127.1.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:09.270399 31924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:09.271880 31924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:09.285389 31938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:09.285346 31934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:09.285642 31935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:09.286244 31924 server_base.cc:1034] running on GCE node
I20260806 22:32:09.286772 31924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:09.287598 31924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:09.288790 31924 hybrid_clock.cc:648] HybridClock initialized: now 1786055529288755 us; error 31 us; skew 500 ppm
I20260806 22:32:09.289053 31924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:09 dist-test-slave-7vjv krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:09.292050 31924 init.cc:377] Logged in from keytab as kudu/127.1.8.66@KRBTEST.COM (short username kudu)
I20260806 22:32:09.293504 31924 webserver.cc:460] Webserver started at http://127.1.8.66:33717/ using document root <none> and password file <none>
I20260806 22:32:09.294116 31924 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:09.294214 31924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:09.294458 31924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:09.297287 31924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "24acb87f879a40508a5f611cc3502f2a"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "07a17ab0ea17a6f6198f5ba2dbc5abae"
server_key_iv: "eacde2809c622330a81026df58b62492"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.298010 31924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "24acb87f879a40508a5f611cc3502f2a"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "07a17ab0ea17a6f6198f5ba2dbc5abae"
server_key_iv: "eacde2809c622330a81026df58b62492"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.303488 31924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:32:09.307407 31952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.308693 31924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:32:09.308856 31924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "24acb87f879a40508a5f611cc3502f2a"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "07a17ab0ea17a6f6198f5ba2dbc5abae"
server_key_iv: "eacde2809c622330a81026df58b62492"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.309046 31924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:09.325724 31924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:09.329432 31924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:09.329717 31924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:09.330646 31924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:09.332021 31924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:09.332118 31924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.332261 31924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:09.332316 31924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.365996 31924 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:35283
I20260806 22:32:09.366046 32131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:35283 every 8 connection(s)
I20260806 22:32:09.367364 31924 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:32:09.371902 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 31924
I20260806 22:32:09.372126 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:32:09.372721 1057 external_mini_cluster.cc:1428] Setting key 2d8b509ac03d8cdc33a57188f1ef8184
Aug 06 22:32:09 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.66@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:09.390219 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.369947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:42923 (local address 127.1.8.126:43869)
0806 22:32:09.370406 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:09.370412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:09.371403 (+ 991us) server_negotiation.cc:407] Connection header received
0806 22:32:09.373908 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:09.373912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:09.373984 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:09.374068 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:09.376044 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.376807 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.379392 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.379831 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.379928 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:09.384240 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:09.384245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:09.384248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:09.384278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:09.386867 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.387587 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.387594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.387600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.387735 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.388218 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.388226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.388231 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.388624 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:09.388787 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:09.389439 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 22:32:09.389687 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":216,"threads_started":1}
I20260806 22:32:09.390926 32142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.370529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43869 (local address 127.1.8.66:42923)
0806 22:32:09.371239 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:09.371276 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:09.373613 (+ 2337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:09.374353 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:09.374364 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:09.375045 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:09.375864 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:09.375892 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.376977 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.376986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:09.379224 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:09.379242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.380142 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.380153 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:09.380476 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:09.381208 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:09.381241 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:09.383995 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:09.387021 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:09.387029 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:09.387045 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:09.387464 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:09.387854 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:09.387860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:09.387863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:09.388055 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:09.388752 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:09.388759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:09.389055 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 22:32:09.389277 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 22:32:09.389667 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":283,"threads_started":1}
I20260806 22:32:09.392416 32132 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43869
I20260806 22:32:09.392761 32132 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:09.393477 32132 heartbeater.cc:511] Master 127.1.8.126:43869 requested a full tablet report, sending...
I20260806 22:32:09.395020 31613 ts_manager.cc:194] Registered new tserver with Master: 24acb87f879a40508a5f611cc3502f2a (127.1.8.66:35283)
I20260806 22:32:09.395836 31613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.66@KRBTEST.COM'} at 127.1.8.66:42923
WARNING: no policy specified for kudu/127.1.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
Entry for principal kudu/127.1.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab.
I20260806 22:32:09.447346 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:09.791560 32147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:09.791859 32147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:09.792057 32147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:09.799137 32147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:09.799289 32147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:09.799458 32147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:32:09.808992 32147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.67.keytab
--principal=kudu/127.1.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:09.810247 32147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:09.811542 32147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:09.825461 32153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:09.825515 32154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:09.825461 32156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:09.826321 32147 server_base.cc:1034] running on GCE node
I20260806 22:32:09.826906 32147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:09.827764 32147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:09.828958 32147 hybrid_clock.cc:648] HybridClock initialized: now 1786055529828925 us; error 24 us; skew 500 ppm
I20260806 22:32:09.829232 32147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:09 dist-test-slave-7vjv krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:09.832537 32147 init.cc:377] Logged in from keytab as kudu/127.1.8.67@KRBTEST.COM (short username kudu)
I20260806 22:32:09.843552 32147 webserver.cc:460] Webserver started at http://127.1.8.67:34737/ using document root <none> and password file <none>
I20260806 22:32:09.844389 32147 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:09.844543 32147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:09.844879 32147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:09.848549 32147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "3dc4d1d47ea7419897fbd856b647a4ec"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "f613c313003b83152291d744864a58b5"
server_key_iv: "2f2d0ca6e4aa6d2375437adad2141ec0"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.849342 32147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "3dc4d1d47ea7419897fbd856b647a4ec"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "f613c313003b83152291d744864a58b5"
server_key_iv: "2f2d0ca6e4aa6d2375437adad2141ec0"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.854487 32147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:09.857887 32170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.859105 32147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:32:09.859285 32147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "3dc4d1d47ea7419897fbd856b647a4ec"
format_stamp: "Formatted at 2026-08-06 22:32:09 on dist-test-slave-7vjv"
server_key: "f613c313003b83152291d744864a58b5"
server_key_iv: "2f2d0ca6e4aa6d2375437adad2141ec0"
server_key_version: "encryptionkey@0"
I20260806 22:32:09.859483 32147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:09.866235 31915 heartbeater.cc:503] Master 127.1.8.126:43869 was elected leader, sending a full tablet report...
I20260806 22:32:09.877528 32147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:09.881868 32147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:09.882167 32147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:09.883134 32147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:09.884533 32147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:09.884639 32147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.884811 32147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:09.884884 32147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:09.918381 32147 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:40423
I20260806 22:32:09.918426 32355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:40423 every 8 connection(s)
I20260806 22:32:09.919713 32147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:32:09.924819 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 32147
I20260806 22:32:09.925017 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:32:09.925594 1057 external_mini_cluster.cc:1428] Setting key dc39e9392a11a93f08bbfd6eac60729f
Aug 06 22:32:09 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.67@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:09.939926 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.922237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:56683 (local address 127.1.8.126:43869)
0806 22:32:09.922714 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:09.922720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:09.923784 (+ 1064us) server_negotiation.cc:407] Connection header received
0806 22:32:09.926235 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:09.926244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:09.926355 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:09.926447 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:09.928283 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.929064 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.930525 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.930900 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.930989 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:09.934344 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:09.934349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:09.934352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:09.934383 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:09.937094 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.937701 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.937704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.937707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.937819 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.938161 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.938164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.938165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.938399 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:09.938523 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:09.939333 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 22:32:09.939574 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":207,"threads_started":1}
I20260806 22:32:09.941066 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.922864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:43869 (local address 127.1.8.67:56683)
0806 22:32:09.923625 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:09.923670 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:09.925928 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:09.926695 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:09.926702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:09.927495 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:09.928068 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:09.928130 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.929199 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.929203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:09.930402 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:09.930415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.931096 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.931100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:09.931271 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:09.931848 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:09.931873 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:09.934169 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:09.937203 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:09.937209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:09.937218 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:09.937596 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:09.937904 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:09.937907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:09.937909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:09.938065 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:09.938497 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:09.938502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:09.938885 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 22:32:09.939178 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 22:32:09.939702 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":303,"threads_started":1}
I20260806 22:32:09.942256 32356 heartbeater.cc:347] Connected to a master server at 127.1.8.126:43869
I20260806 22:32:09.942499 32356 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:09.943218 32356 heartbeater.cc:511] Master 127.1.8.126:43869 requested a full tablet report, sending...
I20260806 22:32:09.944543 31613 ts_manager.cc:194] Registered new tserver with Master: 3dc4d1d47ea7419897fbd856b647a4ec (127.1.8.67:40423)
I20260806 22:32:09.945346 31613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.8.67@KRBTEST.COM'} at 127.1.8.67:56683
I20260806 22:32:09.952849 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:32:09.969552 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.956971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.1.8.126:43869)
0806 22:32:09.957138 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:09.957145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:09.957562 (+ 417us) server_negotiation.cc:407] Connection header received
0806 22:32:09.957755 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:09.957759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:09.957817 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:09.957915 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:09.959285 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.959983 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.961350 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:09.961751 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:09.961889 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:09.964439 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:09.964444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:09.964447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:09.964477 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:09.966788 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.967480 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.967486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.967489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.967631 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:09.968001 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:09.968006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:09.968009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:09.968405 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:09.968697 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:09.969013 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 22:32:09.969241 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:32:09.973335 31613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:32:09.980437 31613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:32:10.015743 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.992831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.1.8.67:40423)
0806 22:32:10.004946 (+ 12115us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.004952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.004983 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:32:10.005064 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.005095 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.005382 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.005589 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:10.006953 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.009204 (+ 2251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.013047 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.014418 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.014568 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.014868 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.015172 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.015334 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11954,"thread_start_us":11787,"threads_started":1}
I20260806 22:32:10.015817 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.992527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:40423 (local address 127.0.0.1:49318)
0806 22:32:10.003718 (+ 11191us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:10.003769 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:10.003962 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:10.005976 (+ 2014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:10.005988 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:10.006043 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:10.006634 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.006653 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.009448 (+ 2795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.009456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.012873 (+ 3417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.012889 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.014635 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.014642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.014862 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.014922 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:32:10.015130 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 22:32:10.015322 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10924,"mutex_wait_us":93,"thread_start_us":12204,"threads_started":1}
I20260806 22:32:10.017212 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.993386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.1.8.66:35283)
0806 22:32:10.004534 (+ 11148us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.004547 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.005153 (+ 606us) server_negotiation.cc:407] Connection header received
0806 22:32:10.005497 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.005508 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.005773 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.005974 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:10.007238 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.009390 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.013993 (+ 4603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.015497 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.015644 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.015848 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.016712 (+ 864us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.016876 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10930,"thread_start_us":10747,"threads_started":1}
I20260806 22:32:10.018117 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:09.992796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:35283 (local address 127.0.0.1:60024)
0806 22:32:10.005030 (+ 12234us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:10.005050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:10.005263 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:10.006356 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:10.006364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:10.006401 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:10.006999 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.007014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.009638 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.009646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.013749 (+ 4103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.013764 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.016260 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.016267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.016462 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.016504 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:32:10.017537 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0806 22:32:10.017685 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12100,"mutex_wait_us":6,"thread_start_us":10509,"threads_started":1}
I20260806 22:32:10.020313 32237 tablet_service.cc:1467] Processing CreateTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (DEFAULT_TABLE table=test-table [id=477c0f725c6f4533a805ef3777c60079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:10.021215 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.005193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:38481 (local address 127.0.0.1:37748)
0806 22:32:10.011726 (+ 6533us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:10.011742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:10.011852 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:10.012987 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:10.012992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:10.013021 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:10.013424 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.013434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.015466 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.015471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.018061 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.018091 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.020461 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.020468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.020641 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.020672 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:32:10.020768 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:32:10.020889 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6410,"thread_start_us":6269,"threads_started":1}
I20260806 22:32:10.021857 32237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a2c16341064c829f3ec5e7e2fdfbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:10.022054 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.005552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.1.8.65:38481)
0806 22:32:10.012436 (+ 6884us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.012445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.012472 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:32:10.012566 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.012573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.012799 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.012996 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:10.013600 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.015277 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.018423 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.019918 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.020049 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.020307 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.021562 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.021731 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6709,"thread_start_us":6498,"threads_started":1}
I20260806 22:32:10.021747 32030 tablet_service.cc:1467] Processing CreateTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (DEFAULT_TABLE table=test-table [id=477c0f725c6f4533a805ef3777c60079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:10.023123 32030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a2c16341064c829f3ec5e7e2fdfbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:10.025502 31807 tablet_service.cc:1467] Processing CreateTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (DEFAULT_TABLE table=test-table [id=477c0f725c6f4533a805ef3777c60079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:10.026863 31807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a2c16341064c829f3ec5e7e2fdfbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:10.034435 32385 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Bootstrap starting.
I20260806 22:32:10.036412 32387 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Bootstrap starting.
I20260806 22:32:10.039454 32385 tablet_bootstrap.cc:654] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:10.039961 32387 tablet_bootstrap.cc:654] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:10.040067 32390 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Bootstrap starting.
I20260806 22:32:10.041055 32385 log.cc:824] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:10.041180 32387 log.cc:824] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:10.043489 32390 tablet_bootstrap.cc:654] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:10.044519 32385 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: No bootstrap required, opened a new log
I20260806 22:32:10.044530 32390 log.cc:824] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:10.044814 32385 ts_tablet_manager.cc:1397] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260806 22:32:10.045399 32387 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: No bootstrap required, opened a new log
I20260806 22:32:10.045674 32387 ts_tablet_manager.cc:1397] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 22:32:10.046976 32390 tablet_bootstrap.cc:492] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: No bootstrap required, opened a new log
I20260806 22:32:10.047219 32390 ts_tablet_manager.cc:1397] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260806 22:32:10.054651 32390 raft_consensus.cc:348] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.055068 32390 raft_consensus.cc:374] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:10.055166 32390 raft_consensus.cc:729] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8efff18118145db84f8810cfec35d93, State: Initialized, Role: FOLLOWER
I20260806 22:32:10.055620 32385 raft_consensus.cc:348] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.055789 32390 consensus_queue.cc:260] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.056026 32385 raft_consensus.cc:374] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:10.056169 32385 raft_consensus.cc:729] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24acb87f879a40508a5f611cc3502f2a, State: Initialized, Role: FOLLOWER
I20260806 22:32:10.056299 32387 raft_consensus.cc:348] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.056707 32387 raft_consensus.cc:374] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:10.056821 32387 raft_consensus.cc:729] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc4d1d47ea7419897fbd856b647a4ec, State: Initialized, Role: FOLLOWER
I20260806 22:32:10.056762 32385 consensus_queue.cc:260] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.057402 32387 consensus_queue.cc:260] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.058357 32132 heartbeater.cc:503] Master 127.1.8.126:43869 was elected leader, sending a full tablet report...
I20260806 22:32:10.058561 32356 heartbeater.cc:503] Master 127.1.8.126:43869 was elected leader, sending a full tablet report...
I20260806 22:32:10.058648 32385 ts_tablet_manager.cc:1428] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 22:32:10.059155 32387 ts_tablet_manager.cc:1428] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 22:32:10.060356 32390 ts_tablet_manager.cc:1428] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Time spent starting tablet: real 0.013s user 0.001s sys 0.010s
I20260806 22:32:10.072382 32394 raft_consensus.cc:482] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:32:10.072603 32394 raft_consensus.cc:504] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.074167 32394 leader_election.cc:288] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3dc4d1d47ea7419897fbd856b647a4ec (127.1.8.67:40423), 24acb87f879a40508a5f611cc3502f2a (127.1.8.66:35283)
I20260806 22:32:10.082643 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.074543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:56947 (local address 127.1.8.67:40423)
0806 22:32:10.074743 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.074751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.075227 (+ 476us) server_negotiation.cc:407] Connection header received
0806 22:32:10.075531 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.075536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.075631 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.075696 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:10.076657 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.077751 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.080930 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.081683 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.081780 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.081849 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.082255 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.082380 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:32:10.082690 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.075108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:40157 (local address 127.1.8.66:35283)
0806 22:32:10.075299 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.075306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.075503 (+ 197us) server_negotiation.cc:407] Connection header received
0806 22:32:10.075661 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.075665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.075717 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.075790 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:10.076623 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.077751 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.080960 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.081762 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.081900 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.081985 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.082319 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.082452 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:32:10.082758 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.074350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:40423 (local address 127.1.8.65:56947)
0806 22:32:10.075102 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:10.075133 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:10.075347 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:10.075886 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:10.075890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:10.075909 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:10.076514 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.076527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.077894 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.077898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.080759 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.080781 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.081905 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.081912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.082095 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.082119 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:32:10.082195 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:32:10.082327 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":216,"thread_start_us":423,"threads_started":1}
I20260806 22:32:10.083128 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.074551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:35283 (local address 127.1.8.65:40157)
0806 22:32:10.075412 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:10.075433 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:10.075542 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:10.075842 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:10.075846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:10.075882 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:10.076428 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.076449 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.077892 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.077897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.080786 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:10.080795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.082004 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.082009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:10.082135 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.082158 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:32:10.082274 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:32:10.082389 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":249,"thread_start_us":753,"threads_started":1}
I20260806 22:32:10.083751 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a2c16341064c829f3ec5e7e2fdfbf1" candidate_uuid: "c8efff18118145db84f8810cfec35d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" is_pre_election: true
I20260806 22:32:10.084079 32065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a2c16341064c829f3ec5e7e2fdfbf1" candidate_uuid: "c8efff18118145db84f8810cfec35d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24acb87f879a40508a5f611cc3502f2a" is_pre_election: true
I20260806 22:32:10.084291 32270 raft_consensus.cc:2454] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8efff18118145db84f8810cfec35d93 in term 0.
I20260806 22:32:10.084609 32065 raft_consensus.cc:2454] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8efff18118145db84f8810cfec35d93 in term 0.
I20260806 22:32:10.085026 31739 leader_election.cc:302] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dc4d1d47ea7419897fbd856b647a4ec, c8efff18118145db84f8810cfec35d93; no voters:
I20260806 22:32:10.085453 32394 raft_consensus.cc:2781] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:32:10.085604 32394 raft_consensus.cc:482] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:32:10.085728 32394 raft_consensus.cc:3037] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:32:10.086521 31917 tablet.cc:2369] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:10.088174 32394 raft_consensus.cc:504] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.088759 32394 leader_election.cc:288] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [CANDIDATE]: Term 1 election: Requested vote from peers 3dc4d1d47ea7419897fbd856b647a4ec (127.1.8.67:40423), 24acb87f879a40508a5f611cc3502f2a (127.1.8.66:35283)
I20260806 22:32:10.089208 32065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a2c16341064c829f3ec5e7e2fdfbf1" candidate_uuid: "c8efff18118145db84f8810cfec35d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24acb87f879a40508a5f611cc3502f2a"
I20260806 22:32:10.089246 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a2c16341064c829f3ec5e7e2fdfbf1" candidate_uuid: "c8efff18118145db84f8810cfec35d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dc4d1d47ea7419897fbd856b647a4ec"
I20260806 22:32:10.089395 32065 raft_consensus.cc:3037] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:10.089418 32270 raft_consensus.cc:3037] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:10.091945 32065 raft_consensus.cc:2454] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8efff18118145db84f8810cfec35d93 in term 1.
I20260806 22:32:10.092008 32270 raft_consensus.cc:2454] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8efff18118145db84f8810cfec35d93 in term 1.
I20260806 22:32:10.092412 31736 leader_election.cc:302] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24acb87f879a40508a5f611cc3502f2a, c8efff18118145db84f8810cfec35d93; no voters:
I20260806 22:32:10.092741 32394 raft_consensus.cc:2781] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:10.093109 32394 raft_consensus.cc:686] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 1 LEADER]: Becoming Leader. State: Replica: c8efff18118145db84f8810cfec35d93, State: Running, Role: LEADER
I20260806 22:32:10.093623 32394 consensus_queue.cc:237] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } }
I20260806 22:32:10.097748 31610 catalog_manager.cc:5495] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8efff18118145db84f8810cfec35d93 (127.1.8.65). New cstate: current_term: 1 leader_uuid: "c8efff18118145db84f8810cfec35d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8efff18118145db84f8810cfec35d93" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 38481 } health_report { overall_health: HEALTHY } } }
W20260806 22:32:10.119350 32134 tablet.cc:2369] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:10.121992 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.116853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.1.8.65:38481)
0806 22:32:10.117026 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.117030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.117047 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:32:10.117193 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.117196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.117268 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.117388 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:32:10.117714 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.118408 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.120173 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.120564 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.120661 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.121119 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:32:10.121575 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:32:10.121679 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.121745 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.121806 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:32:10.129881 32065 raft_consensus.cc:1261] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 1 FOLLOWER]: Refusing update from remote peer c8efff18118145db84f8810cfec35d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:10.129878 32270 raft_consensus.cc:1261] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 1 FOLLOWER]: Refusing update from remote peer c8efff18118145db84f8810cfec35d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:10.130841 32402 consensus_queue.cc:1035] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24acb87f879a40508a5f611cc3502f2a" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:10.131168 32394 consensus_queue.cc:1035] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dc4d1d47ea7419897fbd856b647a4ec" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 40423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:10.145069 32407 mvcc.cc:205] Tried to move back new op lower bound from 7315683451402149888 to 7315683451266650112. Current Snapshot: MvccSnapshot[applied={T|T < 7315683451402149888}]
I20260806 22:32:10.157918 31610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:10.158244 31610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:10.161784 31610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22d9793463154e50bd50cb2254174961: Sending DeleteTablet for 3 replicas of tablet c7a2c16341064c829f3ec5e7e2fdfbf1
I20260806 22:32:10.163008 32030 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:10 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:60024
I20260806 22:32:10.163283 31807 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:10 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:37748
I20260806 22:32:10.163563 32237 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a2c16341064c829f3ec5e7e2fdfbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:10 UTC) from {username='kudu', principal='kudu/127.1.8.126@KRBTEST.COM'} at 127.0.0.1:49318
I20260806 22:32:10.165395 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 31715
I20260806 22:32:10.174131 32420 tablet_replica.cc:330] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: stopping tablet replica
I20260806 22:32:10.174681 32420 raft_consensus.cc:2229] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:32:10.175592 32420 raft_consensus.cc:2258] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:10.177374 32423 tablet_replica.cc:330] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: stopping tablet replica
I20260806 22:32:10.177848 32423 raft_consensus.cc:2229] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:10.178443 32423 raft_consensus.cc:2258] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:10.180763 32423 ts_tablet_manager.cc:1905] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:10.180794 32420 ts_tablet_manager.cc:1905] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:10.187251 32418 tablet_replica.cc:330] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: stopping tablet replica
I20260806 22:32:10.187716 32418 raft_consensus.cc:2229] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:10.188057 32423 ts_tablet_manager.cc:1918] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:10.188226 32423 log.cc:1196] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/c7a2c16341064c829f3ec5e7e2fdfbf1
I20260806 22:32:10.188722 32423 ts_tablet_manager.cc:1939] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 3dc4d1d47ea7419897fbd856b647a4ec: Deleting consensus metadata
I20260806 22:32:10.188953 32420 ts_tablet_manager.cc:1918] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:10.189218 32420 log.cc:1196] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-0/wal/wals/c7a2c16341064c829f3ec5e7e2fdfbf1
I20260806 22:32:10.189810 32420 ts_tablet_manager.cc:1939] T c7a2c16341064c829f3ec5e7e2fdfbf1 P c8efff18118145db84f8810cfec35d93: Deleting consensus metadata
I20260806 22:32:10.190491 31598 catalog_manager.cc:4841] TS 3dc4d1d47ea7419897fbd856b647a4ec (127.1.8.67:40423): tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (table test-table [id=477c0f725c6f4533a805ef3777c60079]) successfully deleted
Aug 06 22:32:10 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.65@KRBTEST.COM
I20260806 22:32:10.191710 31598 catalog_manager.cc:4841] TS c8efff18118145db84f8810cfec35d93 (127.1.8.65:38481): tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (table test-table [id=477c0f725c6f4533a805ef3777c60079]) successfully deleted
I20260806 22:32:10.191946 32418 raft_consensus.cc:2258] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:10.194329 32418 ts_tablet_manager.cc:1905] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:10.201692 32418 ts_tablet_manager.cc:1918] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:10.201912 32418 log.cc:1196] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/c7a2c16341064c829f3ec5e7e2fdfbf1
I20260806 22:32:10.202234 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.166653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.1.8.65:38481)
0806 22:32:10.166898 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.166907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.179512 (+ 12605us) server_negotiation.cc:407] Connection header received
0806 22:32:10.179846 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.179851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.179934 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.180023 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:10.181983 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.183098 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.185094 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.185858 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.185967 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.194194 (+ 8227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:10.194210 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:10.194226 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:10.194272 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:10.198404 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.199403 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.199412 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.199420 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.199628 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.200129 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.200136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.200139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.200629 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:10.200908 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.201531 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.201821 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:32:10.202724 32418 ts_tablet_manager.cc:1939] T c7a2c16341064c829f3ec5e7e2fdfbf1 P 24acb87f879a40508a5f611cc3502f2a: Deleting consensus metadata
I20260806 22:32:10.203802 31905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:10.205154 31595 catalog_manager.cc:4841] TS 24acb87f879a40508a5f611cc3502f2a (127.1.8.66:35283): tablet c7a2c16341064c829f3ec5e7e2fdfbf1 (table test-table [id=477c0f725c6f4533a805ef3777c60079]) successfully deleted
I20260806 22:32:10.300776 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 31715
I20260806 22:32:10.332319 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 31924
Aug 06 22:32:10 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.66@KRBTEST.COM
I20260806 22:32:10.349542 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.333066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.1.8.66:35283)
0806 22:32:10.333253 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.333258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.333276 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:32:10.333409 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.333413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.333491 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.333578 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:10.334962 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.336313 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.338509 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.339206 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.339342 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.344580 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:10.344597 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:10.344624 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:10.344657 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:10.347273 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.347859 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.347865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.347867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.347985 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.348360 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.348363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.348365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.348729 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:10.348937 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.349147 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.349307 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:32:10.350169 32126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:10.438057 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 31924
I20260806 22:32:10.467484 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 32147
Aug 06 22:32:10 dist-test-slave-7vjv krb5kdc[31552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.67@KRBTEST.COM
I20260806 22:32:10.483040 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:10.468355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.1.8.67:40423)
0806 22:32:10.468576 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:10.468584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:10.468669 (+ 85us) server_negotiation.cc:407] Connection header received
0806 22:32:10.468877 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:10.468883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:10.468963 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:10.469090 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:10.470421 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.471182 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.472529 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:10.472925 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:10.473062 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:10.477447 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:10.477461 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:10.477478 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:10.477512 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:10.480039 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.480913 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.480920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.480922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.481058 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:10.481481 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:10.481503 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:10.481505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:10.481850 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:10.482156 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:10.482484 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 22:32:10.482733 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:32:10.483737 32350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:10.572142 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 32147
I20260806 22:32:10.598851 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 31568
I20260806 22:32:10.599982 31659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:10.711576 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 31568
2026-08-06T22:32:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4649 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:32:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:10Z Disabled control of system clock
I20260806 22:32:10.812959 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:44863
--webserver_interface=127.1.8.126
--webserver_port=0
--builtin_ntp_servers=127.1.8.84:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:32:11.150771 32436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:11.151074 32436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:11.151177 32436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:11.158274 32436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:11.158385 32436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:11.158428 32436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:11.158493 32436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:11.158545 32436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:11.167524 32436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:44863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:11.168812 32436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:11.170667 32436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:11.183460 32449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:11.183553 32446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:11.183593 32447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:11.184657 32436 server_base.cc:1034] running on GCE node
I20260806 22:32:11.185376 32436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:11.186705 32436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:11.187906 32436 hybrid_clock.cc:648] HybridClock initialized: now 1786055531187845 us; error 55 us; skew 500 ppm
I20260806 22:32:11.188292 32436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:11.194757 32436 webserver.cc:460] Webserver started at http://127.1.8.126:41113/ using document root <none> and password file <none>
I20260806 22:32:11.195464 32436 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:11.195585 32436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:11.195935 32436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:11.198830 32436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "fa5e66db917042628129a1c057eb464f"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "bf73fa11e99d836b053d4892e38be6c3"
server_key_iv: "af9ac47fa7b9426cb52cd545223460c8"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.199738 32436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "fa5e66db917042628129a1c057eb464f"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "bf73fa11e99d836b053d4892e38be6c3"
server_key_iv: "af9ac47fa7b9426cb52cd545223460c8"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.204768 32436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 22:32:11.208007 32458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:11.209326 32436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:32:11.209509 32436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "fa5e66db917042628129a1c057eb464f"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "bf73fa11e99d836b053d4892e38be6c3"
server_key_iv: "af9ac47fa7b9426cb52cd545223460c8"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.209721 32436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:11.235677 32436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:11.236590 32436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:11.236891 32436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:11.256392 32436 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:44863
I20260806 22:32:11.256412 32541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:44863 every 8 connection(s)
I20260806 22:32:11.257889 32436 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:11.261849 32544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:11.262562 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 32436
I20260806 22:32:11.262749 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:11.263360 1057 external_mini_cluster.cc:1428] Setting key 9559d03bc3b7a9412f1762b8c9a1cce9
I20260806 22:32:11.276038 32544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: Bootstrap starting.
I20260806 22:32:11.277604 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:11.265433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.1.8.126:44863)
0806 22:32:11.266484 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:11.266495 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:11.266543 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:32:11.268482 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:11.268502 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:11.268955 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:11.269273 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:11.269932 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.271207 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.273076 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.273764 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.273903 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:11.275166 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:11.275174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:11.275187 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:11.275711 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:11.275855 (+ 144us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:11.275886 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:11.276011 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:11.276341 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:32:11.276678 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":393,"threads_started":1}
I20260806 22:32:11.279428 32544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:11.280687 32544 log.cc:824] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:11.283432 32544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: No bootstrap required, opened a new log
I20260806 22:32:11.291541 32544 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } }
I20260806 22:32:11.291947 32544 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:11.292057 32544 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa5e66db917042628129a1c057eb464f, State: Initialized, Role: FOLLOWER
I20260806 22:32:11.292683 32544 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } }
I20260806 22:32:11.292936 32544 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:11.293087 32544 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:11.293275 32544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:11.295425 32544 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } }
I20260806 22:32:11.295930 32544 leader_election.cc:302] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa5e66db917042628129a1c057eb464f; no voters:
I20260806 22:32:11.296527 32544 leader_election.cc:288] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:11.296698 32556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:11.297611 32556 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [term 1 LEADER]: Becoming Leader. State: Replica: fa5e66db917042628129a1c057eb464f, State: Running, Role: LEADER
I20260806 22:32:11.298172 32556 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } }
I20260806 22:32:11.298843 32544 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:11.301617 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa5e66db917042628129a1c057eb464f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } } }
I20260806 22:32:11.302049 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:11.301581 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa5e66db917042628129a1c057eb464f. Latest consensus state: current_term: 1 leader_uuid: "fa5e66db917042628129a1c057eb464f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5e66db917042628129a1c057eb464f" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 44863 } } }
I20260806 22:32:11.302688 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:11.304311 32563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:11.310946 32563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:11.324788 32563 catalog_manager.cc:1347] Generated new cluster ID: 9e2b5041a83f4592baf44fc3bfbe5272
I20260806 22:32:11.324939 32563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:11.339422 32563 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:11.341486 32563 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:11.350857 32563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: Generated new TSK 0
I20260806 22:32:11.352041 32563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:11.363283 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.65:0
--local_ip_for_outbound_sockets=127.1.8.65
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--builtin_ntp_servers=127.1.8.84:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:32:11.716472 32582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:11.716785 32582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:11.716938 32582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:11.724143 32582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:11.724274 32582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:11.724457 32582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.65
I20260806 22:32:11.733417 32582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.8.65
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.65
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:11.734627 32582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:11.736421 32582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:11.749781 32591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:11.749866 32594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:11.749836 32589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:11.750762 32582 server_base.cc:1034] running on GCE node
I20260806 22:32:11.751343 32582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:11.752278 32582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:11.753508 32582 hybrid_clock.cc:648] HybridClock initialized: now 1786055531753474 us; error 44 us; skew 500 ppm
I20260806 22:32:11.753821 32582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:11.755960 32582 webserver.cc:460] Webserver started at http://127.1.8.65:40385/ using document root <none> and password file <none>
I20260806 22:32:11.756686 32582 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:11.756819 32582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:11.757154 32582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:11.759974 32582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data/instance:
uuid: "8c6bfc94a88d43619b05f876282d90bb"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "16575ed1cbf676562c14a724cbaaf24e"
server_key_iv: "817d896cca362c3bbabe5c85ef471c57"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.760831 32582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance:
uuid: "8c6bfc94a88d43619b05f876282d90bb"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "16575ed1cbf676562c14a724cbaaf24e"
server_key_iv: "817d896cca362c3bbabe5c85ef471c57"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.765741 32582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:11.769140 32601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:11.770360 32582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:11.770558 32582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal
uuid: "8c6bfc94a88d43619b05f876282d90bb"
format_stamp: "Formatted at 2026-08-06 22:32:11 on dist-test-slave-7vjv"
server_key: "16575ed1cbf676562c14a724cbaaf24e"
server_key_iv: "817d896cca362c3bbabe5c85ef471c57"
server_key_version: "encryptionkey@0"
I20260806 22:32:11.770792 32582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:11.792796 32582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:11.793572 32582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:11.793857 32582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:11.794847 32582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:11.796468 32582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:11.796589 32582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:11.796720 32582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:11.796808 32582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:11.831437 32582 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.65:42767
I20260806 22:32:11.831527 318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.65:42767 every 8 connection(s)
I20260806 22:32:11.832974 32582 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/data/info.pb
I20260806 22:32:11.839452 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 32582
I20260806 22:32:11.839689 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-0/wal/instance
I20260806 22:32:11.840332 1057 external_mini_cluster.cc:1428] Setting key 3c7d74fbe1dc5c7c063e8d0ee180d864
I20260806 22:32:11.843535 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.66:0
--local_ip_for_outbound_sockets=127.1.8.66
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--builtin_ntp_servers=127.1.8.84:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:32:11.847195 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:11.835683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:42097 (local address 127.1.8.126:44863)
0806 22:32:11.836206 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:11.836212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:11.837455 (+ 1243us) server_negotiation.cc:407] Connection header received
0806 22:32:11.839324 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:11.839328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:11.839402 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:11.839501 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:11.841059 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.841902 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.844252 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.844643 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.844739 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:11.846428 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:11.846434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:11.846437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:11.846455 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:11.846497 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:11.846509 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:11.846586 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:11.846866 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:32:11.846944 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":230,"threads_started":1}
I20260806 22:32:11.848694 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:11.836347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:44863 (local address 127.1.8.65:42097)
0806 22:32:11.837272 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:11.837321 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:11.839012 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:11.839822 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:11.839835 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:11.839966 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:11.840852 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:11.840880 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.842096 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.842100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:11.844063 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:11.844121 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:11.844881 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:11.844891 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:11.845251 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:11.846045 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:11.846051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:11.846114 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:11.846223 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:11.846616 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:11.846629 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:11.846640 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:32:11.846836 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 22:32:11.847273 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":289,"threads_started":1}
I20260806 22:32:11.850453 320 heartbeater.cc:347] Connected to a master server at 127.1.8.126:44863
I20260806 22:32:11.850785 320 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:11.851715 320 heartbeater.cc:511] Master 127.1.8.126:44863 requested a full tablet report, sending...
I20260806 22:32:11.853981 32485 ts_manager.cc:194] Registered new tserver with Master: 8c6bfc94a88d43619b05f876282d90bb (127.1.8.65:42767)
I20260806 22:32:11.855486 32485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.65:42097
W20260806 22:32:12.190629 325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:12.190900 325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:12.190987 325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:12.198099 325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:12.198238 325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:12.198350 325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.66
I20260806 22:32:12.207684 325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.8.66
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.66
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:12.209192 325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:12.211052 325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:12.224448 332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:12.224584 333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:12.224807 335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:12.225571 325 server_base.cc:1034] running on GCE node
I20260806 22:32:12.226207 325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:12.227152 325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:12.228358 325 hybrid_clock.cc:648] HybridClock initialized: now 1786055532228318 us; error 31 us; skew 500 ppm
I20260806 22:32:12.228705 325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:12.230981 325 webserver.cc:460] Webserver started at http://127.1.8.66:45367/ using document root <none> and password file <none>
I20260806 22:32:12.231707 325 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:12.231833 325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:12.232229 325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:12.235105 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data/instance:
uuid: "234d1debd3834a88b2f81fb88648a9d0"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "b93ed6f0e500749fe88e11cd85169708"
server_key_iv: "96ecb070d7201a7ae5327c6a1274c287"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.235929 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance:
uuid: "234d1debd3834a88b2f81fb88648a9d0"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "b93ed6f0e500749fe88e11cd85169708"
server_key_iv: "96ecb070d7201a7ae5327c6a1274c287"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.241219 325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 22:32:12.244861 347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.246146 325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:12.246327 325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal
uuid: "234d1debd3834a88b2f81fb88648a9d0"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "b93ed6f0e500749fe88e11cd85169708"
server_key_iv: "96ecb070d7201a7ae5327c6a1274c287"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.246548 325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:12.262250 325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:12.263226 325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:12.263512 325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:12.264743 325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:12.266328 325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:12.266448 325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.266584 325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:12.266680 325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.303120 325 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.66:40545
I20260806 22:32:12.303200 528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.66:40545 every 8 connection(s)
I20260806 22:32:12.304996 325 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/data/info.pb
I20260806 22:32:12.310503 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 325
I20260806 22:32:12.310710 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal/instance
I20260806 22:32:12.311342 1057 external_mini_cluster.cc:1428] Setting key 9314fcdacf2a5eb5c2a43be7af3cbd22
I20260806 22:32:12.316115 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.8.67:0
--local_ip_for_outbound_sockets=127.1.8.67
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--builtin_ntp_servers=127.1.8.84:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:32:12.319716 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.308003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.66:40857 (local address 127.1.8.126:44863)
0806 22:32:12.308275 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.308283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.309718 (+ 1435us) server_negotiation.cc:407] Connection header received
0806 22:32:12.311946 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.311952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.312044 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.312197 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:12.313519 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.314555 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.316683 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.317092 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.317227 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.318861 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:12.318867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:12.318874 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:12.318900 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:12.318933 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:12.318946 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:12.319044 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.319269 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.319369 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:32:12.320827 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.308565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:44863 (local address 127.1.8.66:40857)
0806 22:32:12.309557 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.309603 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.311577 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.312444 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.312453 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.312535 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:12.313309 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.313339 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.314780 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.314788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.316536 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.316549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.317313 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.317321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.317670 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.318535 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:12.318537 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:12.318579 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:12.318654 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:12.319044 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:12.319058 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:12.319069 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.319261 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.319673 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":322,"threads_started":1}
I20260806 22:32:12.322345 529 heartbeater.cc:347] Connected to a master server at 127.1.8.126:44863
I20260806 22:32:12.322654 529 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:12.323364 529 heartbeater.cc:511] Master 127.1.8.126:44863 requested a full tablet report, sending...
I20260806 22:32:12.324963 32485 ts_manager.cc:194] Registered new tserver with Master: 234d1debd3834a88b2f81fb88648a9d0 (127.1.8.66:40545)
I20260806 22:32:12.325920 32485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.66:40857
W20260806 22:32:12.671340 536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:12.671682 536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:12.671830 536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:12.679550 536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:12.679737 536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:12.679927 536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.8.67
I20260806 22:32:12.689218 536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.8.67
--webserver_port=0
--tserver_master_addrs=127.1.8.126:44863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.8.67
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:12.690676 536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:12.692543 536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:12.706024 550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:12.706108 551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:12.706085 555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:12.706935 536 server_base.cc:1034] running on GCE node
I20260806 22:32:12.707617 536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:12.708678 536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:12.709913 536 hybrid_clock.cc:648] HybridClock initialized: now 1786055532709861 us; error 67 us; skew 500 ppm
I20260806 22:32:12.710263 536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:32:12.712864 536 webserver.cc:460] Webserver started at http://127.1.8.67:42761/ using document root <none> and password file <none>
I20260806 22:32:12.713663 536 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:12.713805 536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:12.714176 536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:12.717334 536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data/instance:
uuid: "937ec4664c534421b8afc633d67ed0a6"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "bf7f1ae53654862d0addc0b170581544"
server_key_iv: "f3a80943c5b5c6ecd845b407fcdac5e0"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.718207 536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance:
uuid: "937ec4664c534421b8afc633d67ed0a6"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "bf7f1ae53654862d0addc0b170581544"
server_key_iv: "f3a80943c5b5c6ecd845b407fcdac5e0"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.723699 536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:32:12.727452 565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.729001 536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:32:12.729210 536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal
uuid: "937ec4664c534421b8afc633d67ed0a6"
format_stamp: "Formatted at 2026-08-06 22:32:12 on dist-test-slave-7vjv"
server_key: "bf7f1ae53654862d0addc0b170581544"
server_key_iv: "f3a80943c5b5c6ecd845b407fcdac5e0"
server_key_version: "encryptionkey@0"
I20260806 22:32:12.729457 536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:12.755056 536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:12.756083 536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:12.756453 536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:12.757498 536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:32:12.759063 536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:32:12.759191 536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.759330 536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:32:12.759425 536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:12.794090 536 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.67:36201
I20260806 22:32:12.794154 746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.67:36201 every 8 connection(s)
I20260806 22:32:12.795451 536 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/data/info.pb
I20260806 22:32:12.796556 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 536
I20260806 22:32:12.796757 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal/instance
I20260806 22:32:12.797415 1057 external_mini_cluster.cc:1428] Setting key 955530cf1c7eac0720f7ea9b5a723f6e
I20260806 22:32:12.807935 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.798425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.67:47125 (local address 127.1.8.126:44863)
0806 22:32:12.798652 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.798658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.800695 (+ 2037us) server_negotiation.cc:407] Connection header received
0806 22:32:12.802257 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.802261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.802325 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.802403 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:12.803337 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.804044 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.805534 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.805937 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.806030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.807299 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:12.807304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:12.807307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:12.807323 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:12.807353 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:12.807365 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:12.807434 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.807628 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.807703 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:32:12.808915 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.799060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:44863 (local address 127.1.8.67:47125)
0806 22:32:12.800303 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.800362 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.802037 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.802553 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.802561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.802643 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:12.803188 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.803216 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.804228 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.804232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.805415 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.805426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.806149 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.806154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.806339 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.807020 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:32:12.807023 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:12.807058 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:32:12.807132 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:12.807449 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:12.807457 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:12.807462 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.807593 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.807853 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":335,"threads_started":1}
I20260806 22:32:12.810173 747 heartbeater.cc:347] Connected to a master server at 127.1.8.126:44863
I20260806 22:32:12.810465 747 heartbeater.cc:464] Registering TS with master...
I20260806 22:32:12.811267 747 heartbeater.cc:511] Master 127.1.8.126:44863 requested a full tablet report, sending...
I20260806 22:32:12.812867 32484 ts_manager.cc:194] Registered new tserver with Master: 937ec4664c534421b8afc633d67ed0a6 (127.1.8.67:36201)
I20260806 22:32:12.813766 32484 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.8.67:47125
I20260806 22:32:12.825173 1057 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:32:12.835796 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.829209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.1.8.126:44863)
0806 22:32:12.829405 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.829410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.829947 (+ 537us) server_negotiation.cc:407] Connection header received
0806 22:32:12.830090 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.830094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.830177 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.830272 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:12.830880 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.831905 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.833299 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.833892 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.834067 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.835143 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:12.835150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:12.835155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:12.835174 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:12.835210 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:12.835225 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:12.835312 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.835424 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.835513 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:32:12.839697 32484 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:32:12.846477 32484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:32:12.859133 320 heartbeater.cc:503] Master 127.1.8.126:44863 was elected leader, sending a full tablet report...
I20260806 22:32:12.870389 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.860901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.1.8.66:40545)
0806 22:32:12.862709 (+ 1808us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.862719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.862748 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:32:12.862848 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.862855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.863070 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.863326 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:12.864199 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.865628 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.868087 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.869191 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.869335 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.869620 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.869848 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.870020 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1655,"thread_start_us":1543,"threads_started":1}
I20260806 22:32:12.870407 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.858329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49394)
0806 22:32:12.859225 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.859261 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.859426 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.862139 (+ 2713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.862147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.862186 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:12.862568 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.862578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.865059 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.865064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.867649 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.867664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.869476 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.869483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.869676 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.869714 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.869782 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.870005 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":77,"thread_start_us":742,"threads_started":1}
I20260806 22:32:12.871372 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.860456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40545 (local address 127.0.0.1:47054)
0806 22:32:12.862189 (+ 1733us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.862206 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.862320 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.863608 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.863615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.863640 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:12.864004 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.864019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.865774 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.865779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.867868 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.867894 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.869386 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.869392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.869582 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.869642 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.869793 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.870941 (+ 1148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1603,"mutex_wait_us":4,"thread_start_us":1335,"threads_started":1}
I20260806 22:32:12.871419 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.858543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:36201 (local address 127.0.0.1:34948)
0806 22:32:12.862130 (+ 3587us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.862156 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.863086 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.863992 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.863995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.864015 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:12.864478 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.864490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.866032 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.866037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.868375 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.868389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.870518 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.870524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.870683 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.870718 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.870795 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.870940 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3405,"mutex_wait_us":6,"thread_start_us":1772,"threads_started":1}
I20260806 22:32:12.871954 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.858567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.1.8.65:42767)
0806 22:32:12.861415 (+ 2848us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.861426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.861466 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:12.861590 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.861603 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.861872 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.862764 (+ 892us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:12.863015 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.864912 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.867868 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.869240 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.869928 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.870212 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.870282 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.870485 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2626,"thread_start_us":2522,"threads_started":1}
I20260806 22:32:12.872159 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.858779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.1.8.67:36201)
0806 22:32:12.860886 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.860895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.862277 (+ 1382us) server_negotiation.cc:407] Connection header received
0806 22:32:12.863298 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.863307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.863478 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.863634 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:12.864693 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.865877 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.868576 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.870002 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.870128 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.870378 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.871572 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.871730 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1911,"thread_start_us":2000,"threads_started":1}
I20260806 22:32:12.875286 423 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 (DEFAULT_TABLE table=test-table [id=a4778281f80546c68bf3cf064f70baa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:12.875605 32676 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 (DEFAULT_TABLE table=test-table [id=a4778281f80546c68bf3cf064f70baa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:12.875597 630 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 (DEFAULT_TABLE table=test-table [id=a4778281f80546c68bf3cf064f70baa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:32:12.876943 423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2f026dfb145cea16dacebc47c7da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:12.877063 630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2f026dfb145cea16dacebc47c7da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:12.877056 32676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2f026dfb145cea16dacebc47c7da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:12.888410 779 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Bootstrap starting.
I20260806 22:32:12.891388 780 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Bootstrap starting.
I20260806 22:32:12.891983 779 tablet_bootstrap.cc:654] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:12.892217 783 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Bootstrap starting.
I20260806 22:32:12.893471 779 log.cc:824] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:12.895364 780 tablet_bootstrap.cc:654] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:12.895982 783 tablet_bootstrap.cc:654] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:12.896632 780 log.cc:824] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:12.897212 783 log.cc:824] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:12.898571 779 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: No bootstrap required, opened a new log
I20260806 22:32:12.898964 779 ts_tablet_manager.cc:1397] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 22:32:12.899291 780 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: No bootstrap required, opened a new log
I20260806 22:32:12.899618 780 ts_tablet_manager.cc:1397] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 22:32:12.901422 783 tablet_bootstrap.cc:492] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: No bootstrap required, opened a new log
I20260806 22:32:12.901688 783 ts_tablet_manager.cc:1397] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 22:32:12.907315 780 raft_consensus.cc:348] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.907698 780 raft_consensus.cc:374] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:12.907892 780 raft_consensus.cc:729] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c6bfc94a88d43619b05f876282d90bb, State: Initialized, Role: FOLLOWER
I20260806 22:32:12.908610 780 consensus_queue.cc:260] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.910487 780 ts_tablet_manager.cc:1428] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260806 22:32:12.910297 779 raft_consensus.cc:348] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.910818 779 raft_consensus.cc:374] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:12.910933 779 raft_consensus.cc:729] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234d1debd3834a88b2f81fb88648a9d0, State: Initialized, Role: FOLLOWER
I20260806 22:32:12.911550 779 consensus_queue.cc:260] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.912837 529 heartbeater.cc:503] Master 127.1.8.126:44863 was elected leader, sending a full tablet report...
I20260806 22:32:12.913513 779 ts_tablet_manager.cc:1428] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260806 22:32:12.912349 783 raft_consensus.cc:348] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.914000 783 raft_consensus.cc:374] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:12.914458 783 raft_consensus.cc:729] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937ec4664c534421b8afc633d67ed0a6, State: Initialized, Role: FOLLOWER
I20260806 22:32:12.915072 783 consensus_queue.cc:260] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.916298 747 heartbeater.cc:503] Master 127.1.8.126:44863 was elected leader, sending a full tablet report...
I20260806 22:32:12.916873 783 ts_tablet_manager.cc:1428] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260806 22:32:12.921078 788 raft_consensus.cc:482] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:32:12.921365 788 raft_consensus.cc:504] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.922860 788 leader_election.cc:288] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937ec4664c534421b8afc633d67ed0a6 (127.1.8.67:36201), 234d1debd3834a88b2f81fb88648a9d0 (127.1.8.66:40545)
I20260806 22:32:12.929889 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.923414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:44769 (local address 127.1.8.66:40545)
0806 22:32:12.923671 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.923678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.924010 (+ 332us) server_negotiation.cc:407] Connection header received
0806 22:32:12.924295 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.924299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.924367 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.924476 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:12.925076 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.925809 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.928127 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.928890 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.929031 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.929175 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.929489 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.929631 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 22:32:12.929970 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.923100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.66:40545 (local address 127.1.8.65:44769)
0806 22:32:12.923910 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.923936 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.924138 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.924494 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.924499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.924522 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:12.924932 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.924943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.925943 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.925949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.927844 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.927856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.929170 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.929179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.929322 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.929346 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.929450 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.929586 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":91,"thread_start_us":444,"threads_started":1}
I20260806 22:32:12.930197 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.923312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.8.65:57117 (local address 127.1.8.67:36201)
0806 22:32:12.923500 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.923509 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.924117 (+ 608us) server_negotiation.cc:407] Connection header received
0806 22:32:12.924319 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.924324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.924409 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.924501 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:32:12.925138 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.925910 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.928036 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.928893 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.929031 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.929143 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.929778 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.929918 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:32:12.930991 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.923207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.67:36201 (local address 127.1.8.65:57117)
0806 22:32:12.923997 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:12.924012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:12.924184 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:12.924515 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:12.924519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:12.924541 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:32:12.924983 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.924991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.926051 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.926063 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.927831 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:12.927843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.929102 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.929108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:12.929587 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.929619 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:32:12.930427 (+ 808us) client_negotiation.cc:240] Negotiation successful
0806 22:32:12.930571 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":68,"thread_start_us":456,"threads_started":1}
I20260806 22:32:12.931407 452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2f026dfb145cea16dacebc47c7da9" candidate_uuid: "8c6bfc94a88d43619b05f876282d90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234d1debd3834a88b2f81fb88648a9d0" is_pre_election: true
I20260806 22:32:12.932083 452 raft_consensus.cc:2454] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c6bfc94a88d43619b05f876282d90bb in term 0.
I20260806 22:32:12.931963 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2f026dfb145cea16dacebc47c7da9" candidate_uuid: "8c6bfc94a88d43619b05f876282d90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937ec4664c534421b8afc633d67ed0a6" is_pre_election: true
I20260806 22:32:12.932598 662 raft_consensus.cc:2454] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c6bfc94a88d43619b05f876282d90bb in term 0.
I20260806 22:32:12.932957 32609 leader_election.cc:302] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234d1debd3834a88b2f81fb88648a9d0, 8c6bfc94a88d43619b05f876282d90bb; no voters:
I20260806 22:32:12.933480 788 raft_consensus.cc:2781] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:32:12.933645 788 raft_consensus.cc:482] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:32:12.933753 788 raft_consensus.cc:3037] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:12.936134 788 raft_consensus.cc:504] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.936769 788 leader_election.cc:288] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [CANDIDATE]: Term 1 election: Requested vote from peers 937ec4664c534421b8afc633d67ed0a6 (127.1.8.67:36201), 234d1debd3834a88b2f81fb88648a9d0 (127.1.8.66:40545)
I20260806 22:32:12.937333 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2f026dfb145cea16dacebc47c7da9" candidate_uuid: "8c6bfc94a88d43619b05f876282d90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937ec4664c534421b8afc633d67ed0a6"
I20260806 22:32:12.937333 452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2f026dfb145cea16dacebc47c7da9" candidate_uuid: "8c6bfc94a88d43619b05f876282d90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234d1debd3834a88b2f81fb88648a9d0"
I20260806 22:32:12.937561 662 raft_consensus.cc:3037] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:12.937561 452 raft_consensus.cc:3037] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:12.940897 662 raft_consensus.cc:2454] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c6bfc94a88d43619b05f876282d90bb in term 1.
I20260806 22:32:12.940907 452 raft_consensus.cc:2454] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c6bfc94a88d43619b05f876282d90bb in term 1.
I20260806 22:32:12.941375 32608 leader_election.cc:302] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c6bfc94a88d43619b05f876282d90bb, 937ec4664c534421b8afc633d67ed0a6; no voters:
I20260806 22:32:12.941715 788 raft_consensus.cc:2781] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:12.942248 788 raft_consensus.cc:686] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 1 LEADER]: Becoming Leader. State: Replica: 8c6bfc94a88d43619b05f876282d90bb, State: Running, Role: LEADER
I20260806 22:32:12.942777 788 consensus_queue.cc:237] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } }
I20260806 22:32:12.947683 32484 catalog_manager.cc:5495] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c6bfc94a88d43619b05f876282d90bb (127.1.8.65). New cstate: current_term: 1 leader_uuid: "8c6bfc94a88d43619b05f876282d90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bfc94a88d43619b05f876282d90bb" member_type: VOTER last_known_addr { host: "127.1.8.65" port: 42767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 } health_report { overall_health: UNKNOWN } } }
I20260806 22:32:12.977478 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:12.972473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.1.8.65:42767)
0806 22:32:12.972651 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:12.972655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:12.972672 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:32:12.972762 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:12.972766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:12.972855 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:12.972985 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:32:12.973308 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.974081 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.975498 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:12.975909 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:12.976005 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:12.976530 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:32:12.977022 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:32:12.977133 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:12.977211 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:32:12.977275 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:32:12.986184 452 raft_consensus.cc:1261] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 1 FOLLOWER]: Refusing update from remote peer 8c6bfc94a88d43619b05f876282d90bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:12.986184 662 raft_consensus.cc:1261] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 1 FOLLOWER]: Refusing update from remote peer 8c6bfc94a88d43619b05f876282d90bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:12.987242 795 consensus_queue.cc:1035] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "234d1debd3834a88b2f81fb88648a9d0" member_type: VOTER last_known_addr { host: "127.1.8.66" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:12.987633 788 consensus_queue.cc:1035] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937ec4664c534421b8afc633d67ed0a6" member_type: VOTER last_known_addr { host: "127.1.8.67" port: 36201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:12.999843 799 mvcc.cc:205] Tried to move back new op lower bound from 7315683463100014592 to 7315683462937563136. Current Snapshot: MvccSnapshot[applied={T|T < 7315683463100014592}]
I20260806 22:32:13.015074 32484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:13.015317 32484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:32:13.018826 32484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa5e66db917042628129a1c057eb464f: Sending DeleteTablet for 3 replicas of tablet fdb2f026dfb145cea16dacebc47c7da9
I20260806 22:32:13.020332 423 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:13 UTC) from {username='slave'} at 127.0.0.1:47054
I20260806 22:32:13.020473 630 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:13 UTC) from {username='slave'} at 127.0.0.1:34948
I20260806 22:32:13.021052 32676 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2f026dfb145cea16dacebc47c7da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:32:13 UTC) from {username='slave'} at 127.0.0.1:49394
I20260806 22:32:13.021847 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 32582
I20260806 22:32:13.031530 812 tablet_replica.cc:330] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: stopping tablet replica
I20260806 22:32:13.032135 812 raft_consensus.cc:2229] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:13.034518 812 raft_consensus.cc:2258] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:13.037876 814 tablet_replica.cc:330] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: stopping tablet replica
I20260806 22:32:13.038403 812 ts_tablet_manager.cc:1905] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:13.038420 814 raft_consensus.cc:2229] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:32:13.039189 814 raft_consensus.cc:2258] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:13.043897 814 ts_tablet_manager.cc:1905] T fdb2f026dfb145cea16dacebc47c7da9 P 8c6bfc94a88d43619b05f876282d90bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:13.044410 813 tablet_replica.cc:330] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: stopping tablet replica
I20260806 22:32:13.045147 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.023993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.1.8.65:42767)
0806 22:32:13.024258 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:13.024267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:13.032700 (+ 8433us) server_negotiation.cc:407] Connection header received
0806 22:32:13.033011 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:13.033019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:13.033148 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:13.033236 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:13.034036 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.035387 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.039622 (+ 4235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.040355 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.040466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:13.042882 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:13.042894 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:13.042912 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:13.043747 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:13.043811 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:13.043837 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:13.043935 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:13.044612 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 22:32:13.044732 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:32:13.046504 308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:13.046660 813 raft_consensus.cc:2229] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:32:13.047076 812 ts_tablet_manager.cc:1918] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:13.047308 812 log.cc:1196] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-1/wal/wals/fdb2f026dfb145cea16dacebc47c7da9
I20260806 22:32:13.047647 813 raft_consensus.cc:2258] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:32:13.047955 812 ts_tablet_manager.cc:1939] T fdb2f026dfb145cea16dacebc47c7da9 P 234d1debd3834a88b2f81fb88648a9d0: Deleting consensus metadata
I20260806 22:32:13.049947 32469 catalog_manager.cc:4841] TS 234d1debd3834a88b2f81fb88648a9d0 (127.1.8.66:40545): tablet fdb2f026dfb145cea16dacebc47c7da9 (table test-table [id=a4778281f80546c68bf3cf064f70baa1]) successfully deleted
I20260806 22:32:13.051030 813 ts_tablet_manager.cc:1905] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:32:13.057031 813 ts_tablet_manager.cc:1918] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:32:13.057216 813 log.cc:1196] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Deleting WAL directory at /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055191034056-1057-0/minicluster-data/ts-2/wal/wals/fdb2f026dfb145cea16dacebc47c7da9
I20260806 22:32:13.057677 813 ts_tablet_manager.cc:1939] T fdb2f026dfb145cea16dacebc47c7da9 P 937ec4664c534421b8afc633d67ed0a6: Deleting consensus metadata
I20260806 22:32:13.059293 32468 catalog_manager.cc:4841] TS 937ec4664c534421b8afc633d67ed0a6 (127.1.8.67:36201): tablet fdb2f026dfb145cea16dacebc47c7da9 (table test-table [id=a4778281f80546c68bf3cf064f70baa1]) successfully deleted
I20260806 22:32:13.147147 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 32582
W20260806 22:32:13.174013 32469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.8.65:42767 (error 108)
I20260806 22:32:13.174278 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 325
I20260806 22:32:13.176151 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.175631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49436)
0806 22:32:13.175794 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.175946 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 22:32:13.176513 32469 catalog_manager.cc:4584] TS 8c6bfc94a88d43619b05f876282d90bb (127.1.8.65:42767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdb2f026dfb145cea16dacebc47c7da9: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
I20260806 22:32:13.181561 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.175108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.1.8.66:40545)
0806 22:32:13.175290 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:13.175294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:13.175425 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:32:13.175640 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:13.175646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:13.175760 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:13.175878 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:13.176376 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.177296 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.178480 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.178883 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.178973 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:13.179935 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:13.179944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:13.179959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:13.180736 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:13.180787 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:13.180808 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:13.180905 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:13.181114 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:32:13.181203 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:32:13.182281 519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:13.230365 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.229875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49452)
0806 22:32:13.230095 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.230194 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 22:32:13.231559 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.231115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49458)
0806 22:32:13.231273 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.231340 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:32:13.281036 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 325
I20260806 22:32:13.292483 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.291800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49474)
0806 22:32:13.292083 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.292228 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 22:32:13.293684 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.293194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49490)
0806 22:32:13.293388 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.293510 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 22:32:13.309607 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 536
I20260806 22:32:13.316350 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.310408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.1.8.67:36201)
0806 22:32:13.310570 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:13.310574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:13.310593 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:32:13.310733 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:13.310737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:13.310841 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:13.310927 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:13.311330 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.312194 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.313534 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:13.313920 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:13.314013 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:13.314954 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:13.314963 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:13.314979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:13.315594 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:13.315643 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:13.315663 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:13.315751 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:13.315884 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:32:13.315972 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:32:13.317189 739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:13.374830 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.374323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49492)
0806 22:32:13.374555 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.374661 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 22:32:13.375945 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:13.375548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.65:42767 (local address 127.0.0.1:49500)
0806 22:32:13.375754 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:13.375825 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 22:32:13.406692 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 536
I20260806 22:32:13.431526 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 32436
I20260806 22:32:13.432860 32534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:13.544391 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 32436
2026-08-06T22:32:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2834 ms)
[----------] 6 tests from EncryptionPolicyTest (23324 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:13 dist-test-slave-7vjv krb5kdc[838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:13 dist-test-slave-7vjv krb5kdc[838](info): set up 2 sockets
Aug 06 22:32:13 dist-test-slave-7vjv krb5kdc[838](info): commencing operation
krb5kdc: starting...
W20260806 22:32:14.909390 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:15 dist-test-slave-7vjv krb5kdc[838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:15.272569 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:15.645865 861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:15.646174 861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:15.646356 861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:15.653627 861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:15.653738 861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:15.653849 861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:15.653998 861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:15.654080 861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:15.664963 861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39301
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:15.666437 861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:15.667824 861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:15.680371 869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:15.680394 870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:15.681059 872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:15.681711 861 server_base.cc:1034] running on GCE node
I20260806 22:32:15.682530 861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:15.684060 861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:15.685343 861 hybrid_clock.cc:648] HybridClock initialized: now 1786055535685279 us; error 51 us; skew 500 ppm
I20260806 22:32:15.685722 861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:15 dist-test-slave-7vjv krb5kdc[838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055535, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:15.689015 861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:15.690510 861 webserver.cc:460] Webserver started at http://127.0.0.1:37499/ using document root <none> and password file <none>
I20260806 22:32:15.691256 861 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:15.691408 861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:15.691745 861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:15.694911 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "a56e59da1e5f4544a835abd46aeaec3f"
format_stamp: "Formatted at 2026-08-06 22:32:15 on dist-test-slave-7vjv"
server_key: "151776ef46d10790d4cdda57fe346c72"
server_key_iv: "1e8770a6253bd3552f24f4cd755008a0"
server_key_version: "encryptionkey@0"
I20260806 22:32:15.695710 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "a56e59da1e5f4544a835abd46aeaec3f"
format_stamp: "Formatted at 2026-08-06 22:32:15 on dist-test-slave-7vjv"
server_key: "151776ef46d10790d4cdda57fe346c72"
server_key_iv: "1e8770a6253bd3552f24f4cd755008a0"
server_key_version: "encryptionkey@0"
I20260806 22:32:15.701368 861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 22:32:15.704839 890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:15.706157 861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:15.706351 861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "a56e59da1e5f4544a835abd46aeaec3f"
format_stamp: "Formatted at 2026-08-06 22:32:15 on dist-test-slave-7vjv"
server_key: "151776ef46d10790d4cdda57fe346c72"
server_key_iv: "1e8770a6253bd3552f24f4cd755008a0"
server_key_version: "encryptionkey@0"
I20260806 22:32:15.706532 861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:15.729612 861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:15.733947 861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:15.734277 861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:15.753731 861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39301
I20260806 22:32:15.753767 966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39301 every 8 connection(s)
I20260806 22:32:15.755182 861 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:15.758977 967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:15.765393 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 861
I20260806 22:32:15.765630 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:15.766314 1057 external_mini_cluster.cc:1428] Setting key 3f3d5cc56cfb2dbafee7f07dd41e4658
I20260806 22:32:15.769825 967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f: Bootstrap starting.
I20260806 22:32:15.773538 967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:15.774730 967 log.cc:824] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:15.778056 967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f: No bootstrap required, opened a new log
Aug 06 22:32:15 dist-test-slave-7vjv krb5kdc[838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:32:15.786859 967 raft_consensus.cc:348] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } }
I20260806 22:32:15.787292 967 raft_consensus.cc:374] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:15.787422 967 raft_consensus.cc:729] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56e59da1e5f4544a835abd46aeaec3f, State: Initialized, Role: FOLLOWER
I20260806 22:32:15.788219 967 consensus_queue.cc:260] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } }
I20260806 22:32:15.788503 967 raft_consensus.cc:388] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:15.788671 967 raft_consensus.cc:482] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:15.788856 967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:15.790978 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:15.768343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41567 (local address 127.0.0.1:39301)
0806 22:32:15.769145 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:15.769157 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:15.769199 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:32:15.771020 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:15.771044 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:15.771439 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:15.771787 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:15.773814 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:15.775217 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:15.776988 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:15.777635 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:15.777775 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:15.782815 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:15.782823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:15.782841 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:15.782892 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:15.786331 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:15.787185 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:15.787193 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:15.787199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:15.787453 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:15.787979 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:15.787983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:15.787985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:15.788672 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:15.789047 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:15.789582 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 22:32:15.789933 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":383,"threads_started":1}
I20260806 22:32:15.791572 967 raft_consensus.cc:504] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } }
I20260806 22:32:15.792207 967 leader_election.cc:302] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a56e59da1e5f4544a835abd46aeaec3f; no voters:
I20260806 22:32:15.792917 967 leader_election.cc:288] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:15.793041 974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:15.794395 974 raft_consensus.cc:686] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [term 1 LEADER]: Becoming Leader. State: Replica: a56e59da1e5f4544a835abd46aeaec3f, State: Running, Role: LEADER
I20260806 22:32:15.795224 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:15.795069 974 consensus_queue.cc:237] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } }
I20260806 22:32:15.795563 967 sys_catalog.cc:564] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:15.798184 975 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a56e59da1e5f4544a835abd46aeaec3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } } }
I20260806 22:32:15.798652 975 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:15.803910 976 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a56e59da1e5f4544a835abd46aeaec3f. Latest consensus state: current_term: 1 leader_uuid: "a56e59da1e5f4544a835abd46aeaec3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56e59da1e5f4544a835abd46aeaec3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39301 } } }
I20260806 22:32:15.804366 976 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:15.809196 984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:15.816403 984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:15.828681 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:15.809462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59493 (local address 127.0.0.1:39301)
0806 22:32:15.809667 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:15.809677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:15.810258 (+ 581us) server_negotiation.cc:407] Connection header received
0806 22:32:15.810526 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:15.810532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:15.810645 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:15.810739 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:15.812891 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:15.814151 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:15.816052 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:15.816800 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:15.816904 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:15.820507 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:15.820514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:15.820523 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:15.820565 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:15.824596 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:15.825645 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:15.825650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:15.825654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:15.825803 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:15.826417 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:15.826422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:15.826425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:15.826928 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:15.827199 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:15.828027 (+ 828us) server_negotiation.cc:299] Negotiation successful
0806 22:32:15.828282 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:32:15.837034 984 catalog_manager.cc:1347] Generated new cluster ID: 7000de38e47745eb875025767ca0eb16
I20260806 22:32:15.837147 984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:15.861438 984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:15.863281 984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:15.875995 984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a56e59da1e5f4544a835abd46aeaec3f: Generated new TSK 0
I20260806 22:32:15.876881 984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:15.883320 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 861
I20260806 22:32:15.884639 962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:15.975922 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 861
W20260806 22:32:16.002068 833 connection.cc:535] client connection to 127.0.0.1:39301 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39301): Connection reset by peer (error 104)
2026-08-06T22:32:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:16 dist-test-slave-7vjv krb5kdc[1023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:16 dist-test-slave-7vjv krb5kdc[1023](info): set up 2 sockets
Aug 06 22:32:16 dist-test-slave-7vjv krb5kdc[1023](info): commencing operation
krb5kdc: starting...
W20260806 22:32:17.365851 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:17 dist-test-slave-7vjv krb5kdc[1023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:17.713002 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:18.063005 1048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:18.063321 1048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:18.063416 1048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:18.070549 1048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:18.070665 1048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:18.070717 1048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:18.070770 1048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:18.070814 1048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:18.080756 1048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:18.082063 1048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:18.083346 1048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:18.095299 1055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:18.095338 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:18.095407 1064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:18.096417 1048 server_base.cc:1034] running on GCE node
I20260806 22:32:18.097147 1048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:18.098413 1048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:18.099673 1048 hybrid_clock.cc:648] HybridClock initialized: now 1786055538099624 us; error 43 us; skew 500 ppm
I20260806 22:32:18.100014 1048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:18 dist-test-slave-7vjv krb5kdc[1023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:18.106420 1048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:18.107779 1048 webserver.cc:460] Webserver started at http://127.0.0.1:36939/ using document root <none> and password file <none>
I20260806 22:32:18.108503 1048 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:18.108606 1048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:18.108866 1048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:18.111884 1048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "5dfa1c435ca64d6887a6f56fc4150f93"
format_stamp: "Formatted at 2026-08-06 22:32:18 on dist-test-slave-7vjv"
server_key: "b71b220c64584cf8ff146e6296c00fc3"
server_key_iv: "5d4bf80b1c4aedd79dc16c49087e2f95"
server_key_version: "encryptionkey@0"
I20260806 22:32:18.112717 1048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "5dfa1c435ca64d6887a6f56fc4150f93"
format_stamp: "Formatted at 2026-08-06 22:32:18 on dist-test-slave-7vjv"
server_key: "b71b220c64584cf8ff146e6296c00fc3"
server_key_iv: "5d4bf80b1c4aedd79dc16c49087e2f95"
server_key_version: "encryptionkey@0"
I20260806 22:32:18.117913 1048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:18.121344 1075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:18.122645 1048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:18.122843 1048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "5dfa1c435ca64d6887a6f56fc4150f93"
format_stamp: "Formatted at 2026-08-06 22:32:18 on dist-test-slave-7vjv"
server_key: "b71b220c64584cf8ff146e6296c00fc3"
server_key_iv: "5d4bf80b1c4aedd79dc16c49087e2f95"
server_key_version: "encryptionkey@0"
I20260806 22:32:18.123049 1048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:18.146063 1048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:18.150215 1048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:18.150527 1048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:18.169387 1048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41403
I20260806 22:32:18.169432 1149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41403 every 8 connection(s)
I20260806 22:32:18.170799 1048 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:18.174525 1151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:18.177433 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1048
I20260806 22:32:18.177636 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:18.178133 1057 external_mini_cluster.cc:1428] Setting key 9d3108264e7266d2d53e4448bcea25e9
I20260806 22:32:18.186462 1151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93: Bootstrap starting.
I20260806 22:32:18.190497 1151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:18.191861 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:18.179924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34707 (local address 127.0.0.1:41403)
0806 22:32:18.180756 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:18.180767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:18.180807 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:18.182466 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:18.182488 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:18.182890 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:18.183229 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:18.183819 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:18.185239 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:18.187091 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:18.187726 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:18.187874 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:18.189416 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:18.189424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:18.189437 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:18.189963 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:18.190031 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:18.190054 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:18.190190 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:18.190546 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:32:18.190892 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":443,"threads_started":1}
I20260806 22:32:18.191881 1151 log.cc:824] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:18.195253 1151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93: No bootstrap required, opened a new log
I20260806 22:32:18.195889 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:18.204663 1151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } }
I20260806 22:32:18.205241 1151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:18.205389 1151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dfa1c435ca64d6887a6f56fc4150f93, State: Initialized, Role: FOLLOWER
I20260806 22:32:18.206077 1151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } }
I20260806 22:32:18.206360 1151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:18.206553 1151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:18.206761 1151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:18.207564 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:18.200332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37063 (local address 127.0.0.1:41403)
0806 22:32:18.200529 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:18.200536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:18.200947 (+ 411us) server_negotiation.cc:407] Connection header received
0806 22:32:18.201093 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:18.201099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:18.201184 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:18.201298 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:18.201789 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:18.203005 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:18.204457 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:18.205195 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:18.205352 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:18.206591 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:18.206598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:18.206603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:18.206628 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:18.206700 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:18.206731 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:18.206950 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:18.207116 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:32:18.207178 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:32:18.209657 1151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } }
I20260806 22:32:18.210124 1151 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dfa1c435ca64d6887a6f56fc4150f93; no voters:
I20260806 22:32:18.210675 1151 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:18.210844 1167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:18.211691 1167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [term 1 LEADER]: Becoming Leader. State: Replica: 5dfa1c435ca64d6887a6f56fc4150f93, State: Running, Role: LEADER
I20260806 22:32:18.212348 1167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } }
I20260806 22:32:18.213001 1151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:18.215590 1171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } } }
I20260806 22:32:18.216061 1171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:18.215688 1172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dfa1c435ca64d6887a6f56fc4150f93. Latest consensus state: current_term: 1 leader_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfa1c435ca64d6887a6f56fc4150f93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41403 } } }
I20260806 22:32:18.216657 1172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:18.217586 1188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:18.223095 1188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:18.234460 1188 catalog_manager.cc:1347] Generated new cluster ID: cebeccc48f504869b7284797c9122c08
I20260806 22:32:18.234573 1188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:18.267441 1188 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:18.269476 1188 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:18.277946 1188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dfa1c435ca64d6887a6f56fc4150f93: Generated new TSK 0
I20260806 22:32:18.279052 1188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:18.296149 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1048
W20260806 22:32:18.297580 1144 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34707
W20260806 22:32:18.298372 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1048: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:18.298599 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1048
W20260806 22:32:18.323047 1018 connection.cc:535] client connection to 127.0.0.1:41403 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41403): Connection reset by peer (error 104)
2026-08-06T22:32:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:18 dist-test-slave-7vjv krb5kdc[1212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:18 dist-test-slave-7vjv krb5kdc[1212](info): set up 2 sockets
Aug 06 22:32:18 dist-test-slave-7vjv krb5kdc[1212](info): commencing operation
krb5kdc: starting...
W20260806 22:32:19.663589 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:19 dist-test-slave-7vjv krb5kdc[1212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:19.987582 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:20.333359 1234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:20.333645 1234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:20.333809 1234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:20.343093 1234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:20.343282 1234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:20.343375 1234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:20.343477 1234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:20.343555 1234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:20.354696 1234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:20.356016 1234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:20.357407 1234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:20.369446 1240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:20.369480 1242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:20.370200 1247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:20.370468 1234 server_base.cc:1034] running on GCE node
I20260806 22:32:20.371212 1234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:20.372561 1234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:20.373761 1234 hybrid_clock.cc:648] HybridClock initialized: now 1786055540373723 us; error 29 us; skew 500 ppm
I20260806 22:32:20.374097 1234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:20 dist-test-slave-7vjv krb5kdc[1212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:20.377180 1234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:20.378645 1234 webserver.cc:460] Webserver started at http://127.0.0.1:46361/ using document root <none> and password file <none>
I20260806 22:32:20.379330 1234 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:20.379478 1234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:20.379799 1234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:20.382802 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "392f5584ff2a4e7292657f3222894067"
format_stamp: "Formatted at 2026-08-06 22:32:20 on dist-test-slave-7vjv"
server_key: "7aa8bef1360e51caefde48c0b570ab27"
server_key_iv: "42df387eccd707e59ec29d478be9b6c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:20.383565 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "392f5584ff2a4e7292657f3222894067"
format_stamp: "Formatted at 2026-08-06 22:32:20 on dist-test-slave-7vjv"
server_key: "7aa8bef1360e51caefde48c0b570ab27"
server_key_iv: "42df387eccd707e59ec29d478be9b6c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:20.388644 1234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 22:32:20.392074 1256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:20.393414 1234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:20.393613 1234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "392f5584ff2a4e7292657f3222894067"
format_stamp: "Formatted at 2026-08-06 22:32:20 on dist-test-slave-7vjv"
server_key: "7aa8bef1360e51caefde48c0b570ab27"
server_key_iv: "42df387eccd707e59ec29d478be9b6c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:20.393819 1234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:20.399698 1234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:20.403721 1234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:20.404026 1234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:20.423175 1234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35377
I20260806 22:32:20.423213 1347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35377 every 8 connection(s)
I20260806 22:32:20.424721 1234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:20.428443 1349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:20.430068 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1234
I20260806 22:32:20.430261 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:20.430842 1057 external_mini_cluster.cc:1428] Setting key 508294db1c247be0c5f462ea9f5a810d
I20260806 22:32:20.438444 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:20.432681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33053 (local address 127.0.0.1:35377)
0806 22:32:20.433483 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:20.433494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:20.433535 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:32:20.435122 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:20.435142 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:20.435493 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:20.435811 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:20.436894 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:20.436901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:20.436913 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:20.436926 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:20.436981 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:20.437004 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:20.437176 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:20.437276 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:32:20.437509 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":369,"threads_started":1}
I20260806 22:32:20.442195 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:20.442296 1349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067: Bootstrap starting.
I20260806 22:32:20.446271 1349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:20.447727 1349 log.cc:824] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:20.450245 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:20.447731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39565 (local address 127.0.0.1:35377)
0806 22:32:20.447923 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:20.447929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:20.448278 (+ 349us) server_negotiation.cc:407] Connection header received
0806 22:32:20.448470 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:20.448474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:20.448556 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:20.448661 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:20.449633 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:20.449637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:20.449641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:20.449659 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:20.449704 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:20.449724 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:20.449843 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:20.449913 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:32:20.449969 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:32:20.451152 1349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067: No bootstrap required, opened a new log
I20260806 22:32:20.458639 1349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260806 22:32:20.459076 1349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:20.459199 1349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392f5584ff2a4e7292657f3222894067, State: Initialized, Role: FOLLOWER
I20260806 22:32:20.459800 1349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260806 22:32:20.460037 1349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:20.460199 1349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:20.460383 1349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:20.462656 1349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260806 22:32:20.463182 1349 leader_election.cc:302] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 392f5584ff2a4e7292657f3222894067; no voters:
I20260806 22:32:20.463805 1349 leader_election.cc:288] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:20.463948 1370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:20.464699 1370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [term 1 LEADER]: Becoming Leader. State: Replica: 392f5584ff2a4e7292657f3222894067, State: Running, Role: LEADER
I20260806 22:32:20.465289 1370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260806 22:32:20.466017 1349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:20.469156 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 392f5584ff2a4e7292657f3222894067. Latest consensus state: current_term: 1 leader_uuid: "392f5584ff2a4e7292657f3222894067" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } } }
I20260806 22:32:20.469776 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:20.469434 1372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "392f5584ff2a4e7292657f3222894067" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f5584ff2a4e7292657f3222894067" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } } }
I20260806 22:32:20.469969 1372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:20.470744 1387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:20.476368 1387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:20.485105 1387 catalog_manager.cc:1347] Generated new cluster ID: 3cdafb79327841e2983710c771495771
I20260806 22:32:20.485222 1387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:20.527594 1387 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:20.528080 1387 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:20.533634 1387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 392f5584ff2a4e7292657f3222894067: Generated new TSK 0
I20260806 22:32:20.534569 1387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:20.542773 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1234
W20260806 22:32:20.544286 1342 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33053
W20260806 22:32:20.544994 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1234: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:20.545145 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1234
2026-08-06T22:32:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:20 dist-test-slave-7vjv krb5kdc[1412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:20 dist-test-slave-7vjv krb5kdc[1412](info): set up 2 sockets
Aug 06 22:32:20 dist-test-slave-7vjv krb5kdc[1412](info): commencing operation
krb5kdc: starting...
W20260806 22:32:21.878487 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:22.208547 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:22.565512 1440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:22.565790 1440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:22.565956 1440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:22.573272 1440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:22.573415 1440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:22.573462 1440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:22.573549 1440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:22.573621 1440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:22.583349 1440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:22.584749 1440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:22.586108 1440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:22.597977 1447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:22.598037 1451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:22.598073 1448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:22.598804 1440 server_base.cc:1034] running on GCE node
I20260806 22:32:22.599586 1440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:22.601089 1440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:22.602324 1440 hybrid_clock.cc:648] HybridClock initialized: now 1786055542602277 us; error 53 us; skew 500 ppm
I20260806 22:32:22.602686 1440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:22.605770 1440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:22.607225 1440 webserver.cc:460] Webserver started at http://127.0.0.1:43767/ using document root <none> and password file <none>
I20260806 22:32:22.607928 1440 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:22.608124 1440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:22.608443 1440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:22.611414 1440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "e291a63a7d324a3ba942b1ba8830ad2a"
format_stamp: "Formatted at 2026-08-06 22:32:22 on dist-test-slave-7vjv"
server_key: "b31d781de610316ab4f1c529bcc0db24"
server_key_iv: "bfe09a97660c24708183b701c2a776b9"
server_key_version: "encryptionkey@0"
I20260806 22:32:22.612358 1440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "e291a63a7d324a3ba942b1ba8830ad2a"
format_stamp: "Formatted at 2026-08-06 22:32:22 on dist-test-slave-7vjv"
server_key: "b31d781de610316ab4f1c529bcc0db24"
server_key_iv: "bfe09a97660c24708183b701c2a776b9"
server_key_version: "encryptionkey@0"
I20260806 22:32:22.617488 1440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 22:32:22.620880 1465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:22.622236 1440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:22.622430 1440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "e291a63a7d324a3ba942b1ba8830ad2a"
format_stamp: "Formatted at 2026-08-06 22:32:22 on dist-test-slave-7vjv"
server_key: "b31d781de610316ab4f1c529bcc0db24"
server_key_iv: "bfe09a97660c24708183b701c2a776b9"
server_key_version: "encryptionkey@0"
I20260806 22:32:22.622628 1440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:22.643172 1440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:22.647112 1440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:22.647408 1440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:22.666219 1440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38321
I20260806 22:32:22.666267 1549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38321 every 8 connection(s)
I20260806 22:32:22.667667 1440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:22.671576 1551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:22.672997 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1440
I20260806 22:32:22.673211 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:22.673673 1057 external_mini_cluster.cc:1428] Setting key 99375237cc3a1b409edbef0396eaf10e
I20260806 22:32:22.685969 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a: Bootstrap starting.
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:32:22.689903 1551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:22.691357 1551 log.cc:824] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:22.694777 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a: No bootstrap required, opened a new log
I20260806 22:32:22.697757 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:22.675489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42113 (local address 127.0.0.1:38321)
0806 22:32:22.676337 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:22.676348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:22.676389 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:32:22.677966 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:22.677989 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:22.678394 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:22.678728 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:22.680554 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:22.681821 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:22.683635 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:22.684371 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:22.684494 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:22.689295 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:22.689305 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:22.689325 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:22.689369 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:22.693264 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:22.693945 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:22.693955 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:22.693963 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:22.694177 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:22.694555 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:22.694560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:22.694563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:22.695246 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:22.695573 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:22.695921 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 22:32:22.696381 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":391,"threads_started":1}
I20260806 22:32:22.701520 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:22.703665 1551 raft_consensus.cc:348] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } }
I20260806 22:32:22.704241 1551 raft_consensus.cc:374] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:22.704357 1551 raft_consensus.cc:729] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e291a63a7d324a3ba942b1ba8830ad2a, State: Initialized, Role: FOLLOWER
I20260806 22:32:22.705008 1551 consensus_queue.cc:260] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } }
I20260806 22:32:22.705247 1551 raft_consensus.cc:388] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:22.705401 1551 raft_consensus.cc:482] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:22.705579 1551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:22.708580 1551 raft_consensus.cc:504] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } }
I20260806 22:32:22.709228 1551 leader_election.cc:302] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e291a63a7d324a3ba942b1ba8830ad2a; no voters:
I20260806 22:32:22.709884 1551 leader_election.cc:288] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:22.709959 1566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:22.711387 1566 raft_consensus.cc:686] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [term 1 LEADER]: Becoming Leader. State: Replica: e291a63a7d324a3ba942b1ba8830ad2a, State: Running, Role: LEADER
I20260806 22:32:22.712214 1566 consensus_queue.cc:237] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } }
I20260806 22:32:22.713411 1551 sys_catalog.cc:564] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:22.715380 1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e291a63a7d324a3ba942b1ba8830ad2a. Latest consensus state: current_term: 1 leader_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } } }
I20260806 22:32:22.715366 1568 sys_catalog.cc:455] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291a63a7d324a3ba942b1ba8830ad2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38321 } } }
I20260806 22:32:22.715791 1568 sys_catalog.cc:458] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:22.715791 1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:22.726176 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:22.706170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48919 (local address 127.0.0.1:38321)
0806 22:32:22.706338 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:22.706342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:22.706842 (+ 500us) server_negotiation.cc:407] Connection header received
0806 22:32:22.706958 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:22.706963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:22.707050 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:22.707166 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:22.708614 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:22.709658 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:22.711636 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:22.712480 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:22.712589 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:22.716632 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:22.716642 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:22.716651 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:22.716705 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:22.721585 (+ 4880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:22.722727 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:22.722736 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:22.722742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:22.722960 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:22.723549 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:22.723557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:22.723563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:22.724131 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:22.724441 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:22.725234 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 22:32:22.725580 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:32:22.733412 1576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:22.738864 1576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:22.749162 1576 catalog_manager.cc:1347] Generated new cluster ID: 7a747710eb344b0890fa9c3e4727ca02
I20260806 22:32:22.749306 1576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:22.759814 1576 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:22.761224 1576 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:22.766714 1576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e291a63a7d324a3ba942b1ba8830ad2a: Generated new TSK 0
I20260806 22:32:22.767560 1576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:22.778249 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1440
I20260806 22:32:22.779486 1540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:22.869827 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1440
2026-08-06T22:32:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1603](info): set up 2 sockets
Aug 06 22:32:22 dist-test-slave-7vjv krb5kdc[1603](info): commencing operation
krb5kdc: starting...
W20260806 22:32:24.218719 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:24 dist-test-slave-7vjv krb5kdc[1603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:24.546861 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:24.900293 1624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:24.900665 1624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:24.900835 1624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:24.908459 1624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:24.908618 1624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:24.908725 1624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:24.908835 1624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:24.908958 1624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:24.919239 1624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:24.920703 1624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:24.922092 1624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:24.934342 1641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:24.934463 1642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:24.934367 1644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:24.936008 1624 server_base.cc:1034] running on GCE node
I20260806 22:32:24.936942 1624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:24.938412 1624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:24.939633 1624 hybrid_clock.cc:648] HybridClock initialized: now 1786055544939587 us; error 36 us; skew 500 ppm
I20260806 22:32:24.940019 1624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:24 dist-test-slave-7vjv krb5kdc[1603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:24.947593 1624 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:24.949220 1624 webserver.cc:460] Webserver started at http://127.0.0.1:40857/ using document root <none> and password file <none>
I20260806 22:32:24.949986 1624 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:24.950155 1624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:24.950517 1624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:24.953749 1624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "bb09cd1d27b24f2aabcb4e0e8752344e"
format_stamp: "Formatted at 2026-08-06 22:32:24 on dist-test-slave-7vjv"
server_key: "5375db68a313463b0c68b765ccccee66"
server_key_iv: "5406355683eec90e3461df942446f13c"
server_key_version: "encryptionkey@0"
I20260806 22:32:24.954563 1624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "bb09cd1d27b24f2aabcb4e0e8752344e"
format_stamp: "Formatted at 2026-08-06 22:32:24 on dist-test-slave-7vjv"
server_key: "5375db68a313463b0c68b765ccccee66"
server_key_iv: "5406355683eec90e3461df942446f13c"
server_key_version: "encryptionkey@0"
I20260806 22:32:24.960139 1624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 22:32:24.963609 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:24.964978 1624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 22:32:24.965168 1624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "bb09cd1d27b24f2aabcb4e0e8752344e"
format_stamp: "Formatted at 2026-08-06 22:32:24 on dist-test-slave-7vjv"
server_key: "5375db68a313463b0c68b765ccccee66"
server_key_iv: "5406355683eec90e3461df942446f13c"
server_key_version: "encryptionkey@0"
I20260806 22:32:24.965365 1624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:24.979076 1624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:24.983120 1624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:24.983429 1624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:25.002130 1624 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46673
I20260806 22:32:25.002184 1731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46673 every 8 connection(s)
I20260806 22:32:25.003530 1624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:25.007283 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:25.011308 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1624
I20260806 22:32:25.011515 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:25.011960 1057 external_mini_cluster.cc:1428] Setting key 795ff14289396c1126429d4fe6e6c44c
I20260806 22:32:25.018589 1732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e: Bootstrap starting.
I20260806 22:32:25.021950 1732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:25.023108 1732 log.cc:824] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:25.026072 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:25.013899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42675 (local address 127.0.0.1:46673)
0806 22:32:25.014771 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:25.014783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:25.014823 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:25.017211 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:25.017235 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:25.017627 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:25.018030 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:25.018601 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:25.020008 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:25.021435 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:25.022194 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:25.022440 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:25.023686 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:25.023695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:25.023709 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:25.024370 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:25.024465 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:25.024490 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:25.024625 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:25.024849 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 22:32:25.025144 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":373,"threads_started":1}
I20260806 22:32:25.026336 1732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e: No bootstrap required, opened a new log
I20260806 22:32:25.030056 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:25.035226 1732 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } }
I20260806 22:32:25.035627 1732 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:25.035750 1732 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb09cd1d27b24f2aabcb4e0e8752344e, State: Initialized, Role: FOLLOWER
I20260806 22:32:25.036422 1732 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } }
I20260806 22:32:25.036695 1732 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:25.036854 1732 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:25.037041 1732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:25.039268 1732 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } }
I20260806 22:32:25.039862 1732 leader_election.cc:302] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb09cd1d27b24f2aabcb4e0e8752344e; no voters:
I20260806 22:32:25.040645 1732 leader_election.cc:288] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:25.040781 1754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:25.040894 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:25.034496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33615 (local address 127.0.0.1:46673)
0806 22:32:25.034709 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:25.034717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:25.035098 (+ 381us) server_negotiation.cc:407] Connection header received
0806 22:32:25.035313 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:25.035317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:25.035374 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:25.035481 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:25.036033 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:25.036937 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:25.038285 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:25.038750 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:25.038889 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:25.040017 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:25.040023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:25.040029 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:25.040072 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:25.040147 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:25.040171 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:25.040273 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:25.040418 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:32:25.040507 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:32:25.041412 1754 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [term 1 LEADER]: Becoming Leader. State: Replica: bb09cd1d27b24f2aabcb4e0e8752344e, State: Running, Role: LEADER
I20260806 22:32:25.042253 1754 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } }
I20260806 22:32:25.044162 1732 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:25.045387 1755 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb09cd1d27b24f2aabcb4e0e8752344e. Latest consensus state: current_term: 1 leader_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } } }
I20260806 22:32:25.045424 1756 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb09cd1d27b24f2aabcb4e0e8752344e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46673 } } }
I20260806 22:32:25.045815 1755 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:25.045830 1756 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:25.048166 1762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:25.053366 1762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:25.060353 1781 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:32:25.063154 1762 catalog_manager.cc:1347] Generated new cluster ID: 7ef9a8f09cd74cca902667f2ce61ecbf
I20260806 22:32:25.063256 1762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:25.097729 1762 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:25.099258 1762 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:25.113229 1762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb09cd1d27b24f2aabcb4e0e8752344e: Generated new TSK 0
I20260806 22:32:25.114495 1762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:25.133045 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1624
W20260806 22:32:25.134126 1724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42675
W20260806 22:32:25.134991 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1624: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:25.135164 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1624
2026-08-06T22:32:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:25 dist-test-slave-7vjv krb5kdc[1786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:25 dist-test-slave-7vjv krb5kdc[1786](info): set up 2 sockets
Aug 06 22:32:25 dist-test-slave-7vjv krb5kdc[1786](info): commencing operation
krb5kdc: starting...
W20260806 22:32:26.482282 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:26 dist-test-slave-7vjv krb5kdc[1786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:26.817420 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:27.162377 1809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:27.162719 1809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:27.162890 1809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:27.170169 1809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:27.170329 1809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:27.170432 1809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:27.170552 1809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:27.170632 1809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:27.180910 1809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:27.182281 1809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:27.183600 1809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:27.195631 1819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:27.195789 1820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:27.195698 1824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:27.196749 1809 server_base.cc:1034] running on GCE node
I20260806 22:32:27.197494 1809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:27.198876 1809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:27.200155 1809 hybrid_clock.cc:648] HybridClock initialized: now 1786055547200072 us; error 52 us; skew 500 ppm
I20260806 22:32:27.200507 1809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:27 dist-test-slave-7vjv krb5kdc[1786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:27.203606 1809 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:27.205154 1809 webserver.cc:460] Webserver started at http://127.0.0.1:43549/ using document root <none> and password file <none>
I20260806 22:32:27.205855 1809 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:27.206017 1809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:27.206321 1809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:27.209349 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "c2fb67d987384f70bc9ec4cac8803c18"
format_stamp: "Formatted at 2026-08-06 22:32:27 on dist-test-slave-7vjv"
server_key: "67af0cdb71e7ce4af976ef796d1a74f2"
server_key_iv: "2055dc32ccce9b4942c1a24f430d869c"
server_key_version: "encryptionkey@0"
I20260806 22:32:27.210098 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "c2fb67d987384f70bc9ec4cac8803c18"
format_stamp: "Formatted at 2026-08-06 22:32:27 on dist-test-slave-7vjv"
server_key: "67af0cdb71e7ce4af976ef796d1a74f2"
server_key_iv: "2055dc32ccce9b4942c1a24f430d869c"
server_key_version: "encryptionkey@0"
I20260806 22:32:27.215250 1809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 22:32:27.218729 1834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:27.219960 1809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:32:27.220214 1809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "c2fb67d987384f70bc9ec4cac8803c18"
format_stamp: "Formatted at 2026-08-06 22:32:27 on dist-test-slave-7vjv"
server_key: "67af0cdb71e7ce4af976ef796d1a74f2"
server_key_iv: "2055dc32ccce9b4942c1a24f430d869c"
server_key_version: "encryptionkey@0"
I20260806 22:32:27.220422 1809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:27.226420 1809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:27.230496 1809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:27.230823 1809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:27.250036 1809 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36499
I20260806 22:32:27.250092 1908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36499 every 8 connection(s)
I20260806 22:32:27.251458 1809 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:27.255163 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:27.261237 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1809
I20260806 22:32:27.261480 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:27.262019 1057 external_mini_cluster.cc:1428] Setting key 4d8526f15bcde460d35cc55347305ed8
I20260806 22:32:27.266517 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18: Bootstrap starting.
I20260806 22:32:27.270006 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:27.263874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36583 (local address 127.0.0.1:36499)
0806 22:32:27.264803 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:27.264816 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:27.264856 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:27.266657 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:27.266689 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:27.267033 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:27.267359 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:27.268497 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:27.268506 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:27.268517 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:27.268532 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:27.268591 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:27.268609 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:27.268767 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:27.268906 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:32:27.269111 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":480,"threads_started":1}
I20260806 22:32:27.270318 1910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:27.271440 1910 log.cc:824] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:27.273731 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:27.274591 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18: No bootstrap required, opened a new log
I20260806 22:32:27.281473 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:27.278912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46597 (local address 127.0.0.1:36499)
0806 22:32:27.279122 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:27.279129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:27.279518 (+ 389us) server_negotiation.cc:407] Connection header received
0806 22:32:27.279664 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:27.279669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:27.279726 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:27.279866 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:27.280786 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:27.280793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:27.280798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:27.280817 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:27.280869 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:27.280894 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:27.281029 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:27.281116 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:32:27.281171 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:32:27.284204 1910 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260806 22:32:27.284561 1910 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:27.284646 1910 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2fb67d987384f70bc9ec4cac8803c18, State: Initialized, Role: FOLLOWER
I20260806 22:32:27.285158 1910 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260806 22:32:27.285429 1910 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:27.285586 1910 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:27.285779 1910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:27.288026 1910 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260806 22:32:27.288731 1910 leader_election.cc:302] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2fb67d987384f70bc9ec4cac8803c18; no voters:
I20260806 22:32:27.289337 1910 leader_election.cc:288] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:27.289463 1929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:27.290170 1929 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [term 1 LEADER]: Becoming Leader. State: Replica: c2fb67d987384f70bc9ec4cac8803c18, State: Running, Role: LEADER
I20260806 22:32:27.290752 1929 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260806 22:32:27.291499 1910 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:27.293749 1931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2fb67d987384f70bc9ec4cac8803c18. Latest consensus state: current_term: 1 leader_uuid: "c2fb67d987384f70bc9ec4cac8803c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } } }
I20260806 22:32:27.293851 1930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2fb67d987384f70bc9ec4cac8803c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fb67d987384f70bc9ec4cac8803c18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } } }
I20260806 22:32:27.294188 1931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:27.294260 1930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:27.295226 1936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:27.301872 1936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:27.312666 1936 catalog_manager.cc:1347] Generated new cluster ID: 9cdd5c3aab6a49d2b6110c37aa9c39e0
I20260806 22:32:27.312803 1936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:27.325963 1936 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:27.326426 1936 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:27.337255 1936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2fb67d987384f70bc9ec4cac8803c18: Generated new TSK 0
I20260806 22:32:27.338337 1936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:27.349251 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1809
W20260806 22:32:27.350335 1899 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36583
W20260806 22:32:27.351039 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1809: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:27.351198 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1809
2026-08-06T22:32:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:27 dist-test-slave-7vjv krb5kdc[1962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:27 dist-test-slave-7vjv krb5kdc[1962](info): set up 2 sockets
Aug 06 22:32:27 dist-test-slave-7vjv krb5kdc[1962](info): commencing operation
krb5kdc: starting...
W20260806 22:32:28.733361 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:28 dist-test-slave-7vjv krb5kdc[1962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:29.074916 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:29.427574 1981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:29.427855 1981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:29.428041 1981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:29.434537 1981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:29.434631 1981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:29.434674 1981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:29.434752 1981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:29.434835 1981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:29.444166 1981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:29.445506 1981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:29.446807 1981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:29.458417 1988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:29.458492 1989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:29.458460 1991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:29.459482 1981 server_base.cc:1034] running on GCE node
I20260806 22:32:29.460336 1981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:29.461632 1981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:29.462864 1981 hybrid_clock.cc:648] HybridClock initialized: now 1786055549462829 us; error 41 us; skew 500 ppm
I20260806 22:32:29.463213 1981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:29 dist-test-slave-7vjv krb5kdc[1962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:29.466305 1981 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:29.467856 1981 webserver.cc:460] Webserver started at http://127.0.0.1:33035/ using document root <none> and password file <none>
I20260806 22:32:29.468605 1981 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:29.468739 1981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:29.469035 1981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:29.471845 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "16fb8abde5cc4e969b79ea0da29a1b74"
format_stamp: "Formatted at 2026-08-06 22:32:29 on dist-test-slave-7vjv"
server_key: "322fbb322112b6dd16249ce379314459"
server_key_iv: "8b790a25041424a9d6fd9476fb740993"
server_key_version: "encryptionkey@0"
I20260806 22:32:29.472617 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "16fb8abde5cc4e969b79ea0da29a1b74"
format_stamp: "Formatted at 2026-08-06 22:32:29 on dist-test-slave-7vjv"
server_key: "322fbb322112b6dd16249ce379314459"
server_key_iv: "8b790a25041424a9d6fd9476fb740993"
server_key_version: "encryptionkey@0"
I20260806 22:32:29.477579 1981 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:32:29.481001 2004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:29.482187 1981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:32:29.482367 1981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "16fb8abde5cc4e969b79ea0da29a1b74"
format_stamp: "Formatted at 2026-08-06 22:32:29 on dist-test-slave-7vjv"
server_key: "322fbb322112b6dd16249ce379314459"
server_key_iv: "8b790a25041424a9d6fd9476fb740993"
server_key_version: "encryptionkey@0"
I20260806 22:32:29.482569 1981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:29.504156 1981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:29.508165 1981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:29.508453 1981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:29.526813 1981 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43777
I20260806 22:32:29.526847 2087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43777 every 8 connection(s)
I20260806 22:32:29.528219 1981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:29.530906 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 1981
I20260806 22:32:29.531091 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:29.531570 1057 external_mini_cluster.cc:1428] Setting key 180591180b389cf73c0eb6c9531b6e73
I20260806 22:32:29.531924 2089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:29.546813 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74: Bootstrap starting.
Aug 06 22:32:29 dist-test-slave-7vjv krb5kdc[1962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:32:29.550069 2089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:29.551184 2089 log.cc:824] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:29.554039 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74: No bootstrap required, opened a new log
I20260806 22:32:29.555737 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:29.533350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:33857 (local address 127.0.0.1:43777)
0806 22:32:29.534190 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:29.534201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:29.534242 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:32:29.536460 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:29.536481 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:29.536888 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:29.537222 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:29.539120 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:29.540506 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:29.542503 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:29.543296 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:29.543434 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:29.548527 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:29.548535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:29.548548 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:29.548587 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:29.551684 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:29.552534 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:29.552541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:29.552549 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:29.552712 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:29.553144 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:29.553148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:29.553151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:29.553638 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:29.553974 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:29.554472 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 22:32:29.554789 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":385,"threads_started":1}
I20260806 22:32:29.559437 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:29.562075 2089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } }
I20260806 22:32:29.562462 2089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:29.562568 2089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fb8abde5cc4e969b79ea0da29a1b74, State: Initialized, Role: FOLLOWER
I20260806 22:32:29.563191 2089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } }
I20260806 22:32:29.563455 2089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:29.563604 2089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:29.563835 2089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:29.566406 2089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } }
I20260806 22:32:29.566985 2089 leader_election.cc:302] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16fb8abde5cc4e969b79ea0da29a1b74; no voters:
I20260806 22:32:29.567641 2089 leader_election.cc:288] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:29.567751 2107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:29.569669 2089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:29.574761 2107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [term 1 LEADER]: Becoming Leader. State: Replica: 16fb8abde5cc4e969b79ea0da29a1b74, State: Running, Role: LEADER
I20260806 22:32:29.575279 2107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } }
I20260806 22:32:29.578277 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16fb8abde5cc4e969b79ea0da29a1b74. Latest consensus state: current_term: 1 leader_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } } }
I20260806 22:32:29.578713 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:29.578331 2109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16fb8abde5cc4e969b79ea0da29a1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43777 } } }
I20260806 22:32:29.579254 2109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:29.583034 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:29.563983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:45667 (local address 127.0.0.1:43777)
0806 22:32:29.564167 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:29.564173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:29.564626 (+ 453us) server_negotiation.cc:407] Connection header received
0806 22:32:29.564757 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:29.564762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:29.564844 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:29.564944 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:29.566341 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:29.567520 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:29.569865 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:29.570816 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:29.570948 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:29.574873 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:29.574878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:29.574908 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:29.574947 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:29.578555 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:29.579555 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:29.579561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:29.579564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:29.579738 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:29.580525 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:29.580533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:29.580539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:29.581050 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:29.581402 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:29.582256 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 22:32:29.582522 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:32:29.588382 2129 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:32:29.589039 2131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:29.592995 2131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:29.601593 2131 catalog_manager.cc:1347] Generated new cluster ID: 11605855d9a24c9cb9bb7dbe906e5b20
I20260806 22:32:29.601742 2131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:29.610740 2131 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:29.611907 2131 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:29.623436 2131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16fb8abde5cc4e969b79ea0da29a1b74: Generated new TSK 0
I20260806 22:32:29.624323 2131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:29.630878 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 1981
I20260806 22:32:29.631989 2082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:29.722425 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 1981
W20260806 22:32:29.749307 1959 connection.cc:535] client connection to 127.0.0.1:43777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43777): Connection reset by peer (error 104)
2026-08-06T22:32:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:29 dist-test-slave-7vjv krb5kdc[2141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:29 dist-test-slave-7vjv krb5kdc[2141](info): set up 2 sockets
Aug 06 22:32:29 dist-test-slave-7vjv krb5kdc[2141](info): commencing operation
krb5kdc: starting...
W20260806 22:32:31.069060 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.005s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:31 dist-test-slave-7vjv krb5kdc[2141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:31.430950 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:31.790329 2163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:31.790649 2163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:31.790814 2163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:31.798024 2163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:31.798168 2163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:31.798273 2163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:31.798380 2163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:31.798455 2163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:31.808316 2163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:31.809669 2163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:31.810957 2163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:31.822723 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:31.822755 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:31.822814 2176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:31.823611 2163 server_base.cc:1034] running on GCE node
I20260806 22:32:31.824476 2163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:31.826045 2163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:31.827267 2163 hybrid_clock.cc:648] HybridClock initialized: now 1786055551827211 us; error 50 us; skew 500 ppm
I20260806 22:32:31.827634 2163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:31 dist-test-slave-7vjv krb5kdc[2141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055551, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:31.835388 2163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:31.836884 2163 webserver.cc:460] Webserver started at http://127.0.0.1:41461/ using document root <none> and password file <none>
I20260806 22:32:31.837587 2163 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:31.837692 2163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:31.837953 2163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:31.841039 2163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "ae13199e358a4b7f9681208824618a5d"
format_stamp: "Formatted at 2026-08-06 22:32:31 on dist-test-slave-7vjv"
server_key: "ce64e3325ce99d8411a37718a254ec60"
server_key_iv: "daab07f8fceee7bb7e81664a238b1d3e"
server_key_version: "encryptionkey@0"
I20260806 22:32:31.841732 2163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "ae13199e358a4b7f9681208824618a5d"
format_stamp: "Formatted at 2026-08-06 22:32:31 on dist-test-slave-7vjv"
server_key: "ce64e3325ce99d8411a37718a254ec60"
server_key_iv: "daab07f8fceee7bb7e81664a238b1d3e"
server_key_version: "encryptionkey@0"
I20260806 22:32:31.846812 2163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:32:31.850194 2192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:31.851465 2163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:32:31.851599 2163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "ae13199e358a4b7f9681208824618a5d"
format_stamp: "Formatted at 2026-08-06 22:32:31 on dist-test-slave-7vjv"
server_key: "ce64e3325ce99d8411a37718a254ec60"
server_key_iv: "daab07f8fceee7bb7e81664a238b1d3e"
server_key_version: "encryptionkey@0"
I20260806 22:32:31.851759 2163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:31.864006 2163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:31.868543 2163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:31.868785 2163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:31.887100 2163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42761
I20260806 22:32:31.887138 2293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42761 every 8 connection(s)
I20260806 22:32:31.888566 2163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:31.892364 2294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:31.896533 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2163
I20260806 22:32:31.896739 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:31.897193 1057 external_mini_cluster.cc:1428] Setting key e44ec91876c3b7ae3b895d32887ec64a
I20260806 22:32:31.905014 2294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: Bootstrap starting.
I20260806 22:32:31.908793 2294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:31.910084 2294 log.cc:824] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:31.913033 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:31.899036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:35569 (local address 127.0.0.1:42761)
0806 22:32:31.899834 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:31.899850 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:31.899902 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:32:31.903023 (+ 3121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:31.903063 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:31.903569 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:31.904018 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:31.904591 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:31.906593 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:31.907947 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:31.909068 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:31.909302 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:31.910411 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:31.910427 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:31.910440 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:31.911243 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:31.911348 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:31.911386 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:31.911568 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:31.911848 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:32:31.912031 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":305,"threads_started":1}
I20260806 22:32:31.913269 2294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: No bootstrap required, opened a new log
I20260806 22:32:31.917004 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:31.921815 2294 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260806 22:32:31.922284 2294 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:31.922400 2294 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae13199e358a4b7f9681208824618a5d, State: Initialized, Role: FOLLOWER
I20260806 22:32:31.923053 2294 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260806 22:32:31.923380 2294 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:31.923538 2294 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:31.923738 2294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:31.926291 2294 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260806 22:32:31.926939 2294 leader_election.cc:302] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae13199e358a4b7f9681208824618a5d; no voters:
I20260806 22:32:31.928004 2294 leader_election.cc:288] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:31.928192 2309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:31.928537 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:31.921506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:43429 (local address 127.0.0.1:42761)
0806 22:32:31.921689 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:31.921695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:31.922100 (+ 405us) server_negotiation.cc:407] Connection header received
0806 22:32:31.922251 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:31.922259 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:31.922346 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:31.922463 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:31.923078 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:31.924234 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:31.925575 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:31.926054 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:31.926184 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:31.927479 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:31.927484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:31.927487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:31.927505 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:31.927542 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:31.927559 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:31.927678 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:31.927955 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:32:31.928032 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:32:31.930490 2294 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:31.935783 2309 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [term 1 LEADER]: Becoming Leader. State: Replica: ae13199e358a4b7f9681208824618a5d, State: Running, Role: LEADER
I20260806 22:32:31.936416 2309 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260806 22:32:31.939301 2314 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae13199e358a4b7f9681208824618a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260806 22:32:31.939730 2314 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:31.939646 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae13199e358a4b7f9681208824618a5d. Latest consensus state: current_term: 1 leader_uuid: "ae13199e358a4b7f9681208824618a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae13199e358a4b7f9681208824618a5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260806 22:32:31.939915 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d [sys.catalog]: This master's current role is: LEADER
W20260806 22:32:31.945685 2333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:32:31.945822 2333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:32:31.945931 2335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:31.948050 2335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:31.957265 2335 catalog_manager.cc:1347] Generated new cluster ID: fd49ca0de4324985823e2326fb2a8447
I20260806 22:32:31.957424 2335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:31.975212 2335 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:31.976614 2335 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:31.991092 2335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae13199e358a4b7f9681208824618a5d: Generated new TSK 0
I20260806 22:32:31.991999 2335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:32.003577 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2163
W20260806 22:32:32.004731 2282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:35569
W20260806 22:32:32.005573 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:32.005741 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2163
W20260806 22:32:32.037150 2138 connection.cc:535] client connection to 127.0.0.1:42761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42761): Connection reset by peer (error 104)
2026-08-06T22:32:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:32 dist-test-slave-7vjv krb5kdc[2348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:32 dist-test-slave-7vjv krb5kdc[2348](info): set up 2 sockets
Aug 06 22:32:32 dist-test-slave-7vjv krb5kdc[2348](info): commencing operation
krb5kdc: starting...
W20260806 22:32:33.373466 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:33 dist-test-slave-7vjv krb5kdc[2348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:33.727662 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:34.082365 2369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:34.082669 2369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:34.082825 2369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:34.090260 2369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:34.090389 2369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:34.090490 2369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:34.090571 2369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:34.090644 2369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:34.100988 2369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:34.102370 2369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:34.103718 2369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:34.116029 2374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:34.116117 2375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:34.116117 2377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:34.117328 2369 server_base.cc:1034] running on GCE node
I20260806 22:32:34.118137 2369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:34.119580 2369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:34.120824 2369 hybrid_clock.cc:648] HybridClock initialized: now 1786055554120779 us; error 54 us; skew 500 ppm
I20260806 22:32:34.121189 2369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:34 dist-test-slave-7vjv krb5kdc[2348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:34.124476 2369 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:34.125963 2369 webserver.cc:460] Webserver started at http://127.0.0.1:37475/ using document root <none> and password file <none>
I20260806 22:32:34.126677 2369 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:34.126824 2369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:34.127159 2369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:34.130206 2369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "b73a4050672040119e89bdab478b8e32"
format_stamp: "Formatted at 2026-08-06 22:32:34 on dist-test-slave-7vjv"
server_key: "9a228b13c0b544d08b4be01d39cdfda7"
server_key_iv: "85a4f6ced4812614c000899ae1b4e157"
server_key_version: "encryptionkey@0"
I20260806 22:32:34.130998 2369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "b73a4050672040119e89bdab478b8e32"
format_stamp: "Formatted at 2026-08-06 22:32:34 on dist-test-slave-7vjv"
server_key: "9a228b13c0b544d08b4be01d39cdfda7"
server_key_iv: "85a4f6ced4812614c000899ae1b4e157"
server_key_version: "encryptionkey@0"
I20260806 22:32:34.136396 2369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 22:32:34.140064 2389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:34.141585 2369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:34.141790 2369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "b73a4050672040119e89bdab478b8e32"
format_stamp: "Formatted at 2026-08-06 22:32:34 on dist-test-slave-7vjv"
server_key: "9a228b13c0b544d08b4be01d39cdfda7"
server_key_iv: "85a4f6ced4812614c000899ae1b4e157"
server_key_version: "encryptionkey@0"
I20260806 22:32:34.142025 2369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:34.147975 2369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:34.152823 2369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:34.153178 2369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:34.172950 2369 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33941
I20260806 22:32:34.173048 2473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33941 every 8 connection(s)
I20260806 22:32:34.174432 2369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:34.178311 2474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:34.183051 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2369
I20260806 22:32:34.183255 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:34.183818 1057 external_mini_cluster.cc:1428] Setting key b008a139ea9f6efaa161ca3713e7d78d
I20260806 22:32:34.190353 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32: Bootstrap starting.
I20260806 22:32:34.192332 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:34.185829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:50111 (local address 127.0.0.1:33941)
0806 22:32:34.186614 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:34.186627 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:34.186666 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:32:34.188862 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:34.188886 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:34.189220 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:34.189609 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:34.190643 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:34.190649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:34.190662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:34.190675 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:34.190733 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:34.190751 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:34.190991 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:34.191126 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:32:34.191333 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":343,"threads_started":1}
I20260806 22:32:34.194147 2474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:34.195473 2474 log.cc:824] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:34.196327 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:34.198959 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32: No bootstrap required, opened a new log
I20260806 22:32:34.204231 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:34.201739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:48537 (local address 127.0.0.1:33941)
0806 22:32:34.201929 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:34.201935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:34.202314 (+ 379us) server_negotiation.cc:407] Connection header received
0806 22:32:34.202453 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:34.202458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:34.202525 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:34.202627 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:34.203556 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:34.203563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:34.203573 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:34.203592 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:34.203645 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:34.203665 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:34.203797 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:34.203849 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:32:34.203907 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:32:34.208226 2474 raft_consensus.cc:348] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } }
I20260806 22:32:34.208597 2474 raft_consensus.cc:374] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:34.208678 2474 raft_consensus.cc:729] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73a4050672040119e89bdab478b8e32, State: Initialized, Role: FOLLOWER
I20260806 22:32:34.209276 2474 consensus_queue.cc:260] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } }
I20260806 22:32:34.209589 2474 raft_consensus.cc:388] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:34.209753 2474 raft_consensus.cc:482] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:34.209936 2474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:34.212231 2474 raft_consensus.cc:504] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } }
I20260806 22:32:34.212778 2474 leader_election.cc:302] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b73a4050672040119e89bdab478b8e32; no voters:
I20260806 22:32:34.213398 2474 leader_election.cc:288] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:34.213557 2491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:34.214295 2491 raft_consensus.cc:686] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [term 1 LEADER]: Becoming Leader. State: Replica: b73a4050672040119e89bdab478b8e32, State: Running, Role: LEADER
I20260806 22:32:34.214859 2491 consensus_queue.cc:237] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } }
I20260806 22:32:34.215672 2474 sys_catalog.cc:564] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:34.218217 2492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b73a4050672040119e89bdab478b8e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } } }
I20260806 22:32:34.218714 2492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:34.219484 2493 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b73a4050672040119e89bdab478b8e32. Latest consensus state: current_term: 1 leader_uuid: "b73a4050672040119e89bdab478b8e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73a4050672040119e89bdab478b8e32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33941 } } }
I20260806 22:32:34.219966 2493 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:34.220340 2506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:34.225239 2506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:34.233636 2506 catalog_manager.cc:1347] Generated new cluster ID: 05e6d5dd1a8543b8b1668417948a4214
I20260806 22:32:34.233767 2506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:34.269284 2506 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:34.269758 2506 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:34.277464 2506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b73a4050672040119e89bdab478b8e32: Generated new TSK 0
I20260806 22:32:34.278313 2506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:34.286479 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2369
W20260806 22:32:34.287549 2466 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:50111
W20260806 22:32:34.288235 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:34.288373 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2369
2026-08-06T22:32:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:34 dist-test-slave-7vjv krb5kdc[2532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:34 dist-test-slave-7vjv krb5kdc[2532](info): set up 2 sockets
Aug 06 22:32:34 dist-test-slave-7vjv krb5kdc[2532](info): commencing operation
krb5kdc: starting...
W20260806 22:32:35.660009 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:35 dist-test-slave-7vjv krb5kdc[2532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:36.013069 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:36.379590 2551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:36.379961 2551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:36.380141 2551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:36.387416 2551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:36.387576 2551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:36.387682 2551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:36.387799 2551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:36.387884 2551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:36.397833 2551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:36.399300 2551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:36.400702 2551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:36.413631 2559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:36.413579 2565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:36.413542 2562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:36.415232 2551 server_base.cc:1034] running on GCE node
I20260806 22:32:36.416178 2551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:36.417657 2551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:36.418923 2551 hybrid_clock.cc:648] HybridClock initialized: now 1786055556418882 us; error 44 us; skew 500 ppm
I20260806 22:32:36.419283 2551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:36 dist-test-slave-7vjv krb5kdc[2532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055556, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:36.422505 2551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:36.424052 2551 webserver.cc:460] Webserver started at http://127.0.0.1:34009/ using document root <none> and password file <none>
I20260806 22:32:36.424799 2551 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:36.424949 2551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:36.425243 2551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:36.428287 2551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "b9a11885a10f484699b61a60a3516135"
format_stamp: "Formatted at 2026-08-06 22:32:36 on dist-test-slave-7vjv"
server_key: "96083d375a1154109c8510b55a869dd9"
server_key_iv: "7c83f09a7a07ca78d7b6e59585e1fe4b"
server_key_version: "encryptionkey@0"
I20260806 22:32:36.429069 2551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "b9a11885a10f484699b61a60a3516135"
format_stamp: "Formatted at 2026-08-06 22:32:36 on dist-test-slave-7vjv"
server_key: "96083d375a1154109c8510b55a869dd9"
server_key_iv: "7c83f09a7a07ca78d7b6e59585e1fe4b"
server_key_version: "encryptionkey@0"
I20260806 22:32:36.434160 2551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 22:32:36.437528 2576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:36.438808 2551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:32:36.438966 2551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "b9a11885a10f484699b61a60a3516135"
format_stamp: "Formatted at 2026-08-06 22:32:36 on dist-test-slave-7vjv"
server_key: "96083d375a1154109c8510b55a869dd9"
server_key_iv: "7c83f09a7a07ca78d7b6e59585e1fe4b"
server_key_version: "encryptionkey@0"
I20260806 22:32:36.439152 2551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:36.462374 2551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:36.466657 2551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:36.466931 2551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:36.484519 2551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43665
I20260806 22:32:36.484548 2658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43665 every 8 connection(s)
I20260806 22:32:36.486050 2551 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:36.489828 2659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:36.492409 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2551
I20260806 22:32:36.492609 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:36.493069 1057 external_mini_cluster.cc:1428] Setting key bc22171d703b7e3ab6af3a9f70acb7f3
I20260806 22:32:36.502223 2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135: Bootstrap starting.
I20260806 22:32:36.506304 2659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:36.507792 2659 log.cc:824] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:32:36 dist-test-slave-7vjv krb5kdc[2532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:32:36.511461 2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135: No bootstrap required, opened a new log
I20260806 22:32:36.519088 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:36.494905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:56377 (local address 127.0.0.1:43665)
0806 22:32:36.495737 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:36.495749 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:36.495791 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:32:36.497469 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:36.497493 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:36.497896 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:36.498277 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:36.500249 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:36.501600 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:36.503436 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:36.504300 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:36.504469 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:36.509893 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:36.509904 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:36.509924 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:36.509970 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:36.514121 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:36.514930 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:36.514944 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:36.514960 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:36.515207 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:36.515687 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:36.515692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:36.515696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:36.516441 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:36.516783 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:36.517242 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 22:32:36.517649 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":395,"threads_started":1}
I20260806 22:32:36.522379 2659 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } }
I20260806 22:32:36.522764 2659 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:36.522886 2659 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a11885a10f484699b61a60a3516135, State: Initialized, Role: FOLLOWER
I20260806 22:32:36.523442 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:36.523573 2659 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } }
I20260806 22:32:36.523829 2659 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:36.523975 2659 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:36.524169 2659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:36.526479 2659 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } }
I20260806 22:32:36.527070 2659 leader_election.cc:302] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a11885a10f484699b61a60a3516135; no voters:
I20260806 22:32:36.527661 2659 leader_election.cc:288] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:36.527817 2672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:36.530196 2659 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:36.532374 2672 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [term 1 LEADER]: Becoming Leader. State: Replica: b9a11885a10f484699b61a60a3516135, State: Running, Role: LEADER
I20260806 22:32:36.532977 2672 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } }
I20260806 22:32:36.535593 2676 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a11885a10f484699b61a60a3516135. Latest consensus state: current_term: 1 leader_uuid: "b9a11885a10f484699b61a60a3516135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } } }
I20260806 22:32:36.536043 2676 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:36.535701 2675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a11885a10f484699b61a60a3516135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a11885a10f484699b61a60a3516135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43665 } } }
I20260806 22:32:36.536670 2675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:36.539248 2685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:36.546108 2685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:36.557106 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:36.528766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:51905 (local address 127.0.0.1:43665)
0806 22:32:36.528929 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:36.528936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:36.535233 (+ 6297us) server_negotiation.cc:407] Connection header received
0806 22:32:36.535476 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:36.535485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:36.535603 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:36.535761 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:36.538397 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:36.539772 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:36.541499 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:36.542231 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:36.542394 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:36.546156 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:36.546166 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:36.546172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:36.546215 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:36.552573 (+ 6358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:36.553802 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:36.553809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:36.553814 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:36.554004 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:36.554664 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:36.554669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:36.554672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:36.555082 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:36.555370 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:36.556558 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0806 22:32:36.556752 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:32:36.570925 2685 catalog_manager.cc:1347] Generated new cluster ID: ba63b72fcb8143508bd1ee69e62a5985
I20260806 22:32:36.571098 2685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:36.583242 2685 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:36.584744 2685 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:36.598804 2685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a11885a10f484699b61a60a3516135: Generated new TSK 0
I20260806 22:32:36.599766 2685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:36.611654 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2551
I20260806 22:32:36.612993 2652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:36.707310 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2551
W20260806 22:32:36.735440 2530 connection.cc:535] client connection to 127.0.0.1:43665 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43665): Connection reset by peer (error 104)
2026-08-06T22:32:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:36 dist-test-slave-7vjv krb5kdc[2706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:36 dist-test-slave-7vjv krb5kdc[2706](info): set up 2 sockets
Aug 06 22:32:36 dist-test-slave-7vjv krb5kdc[2706](info): commencing operation
krb5kdc: starting...
W20260806 22:32:38.067802 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:38 dist-test-slave-7vjv krb5kdc[2706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:38.427786 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:38.795897 2736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:38.796206 2736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:38.796301 2736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:38.803292 2736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:38.803395 2736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:38.803438 2736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:38.803490 2736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:38.803529 2736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:38.813139 2736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:38.814320 2736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:38.815573 2736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:38.827610 2744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:38.827551 2750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:38.827656 2748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:38.828812 2736 server_base.cc:1034] running on GCE node
I20260806 22:32:38.829600 2736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:38.830948 2736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:38.832185 2736 hybrid_clock.cc:648] HybridClock initialized: now 1786055558832146 us; error 32 us; skew 500 ppm
I20260806 22:32:38.832538 2736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:38 dist-test-slave-7vjv krb5kdc[2706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055558, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:38.835644 2736 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:38.837136 2736 webserver.cc:460] Webserver started at http://127.0.0.1:46145/ using document root <none> and password file <none>
I20260806 22:32:38.837761 2736 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:38.837864 2736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:38.838120 2736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:38.841100 2736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92"
format_stamp: "Formatted at 2026-08-06 22:32:38 on dist-test-slave-7vjv"
server_key: "b23ebf3a3d03e2d345e4112a86456b1c"
server_key_iv: "c5590c2a42439daebf109d68eb32633f"
server_key_version: "encryptionkey@0"
I20260806 22:32:38.841825 2736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92"
format_stamp: "Formatted at 2026-08-06 22:32:38 on dist-test-slave-7vjv"
server_key: "b23ebf3a3d03e2d345e4112a86456b1c"
server_key_iv: "c5590c2a42439daebf109d68eb32633f"
server_key_version: "encryptionkey@0"
I20260806 22:32:38.846868 2736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 22:32:38.850226 2764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:38.851542 2736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:32:38.851723 2736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92"
format_stamp: "Formatted at 2026-08-06 22:32:38 on dist-test-slave-7vjv"
server_key: "b23ebf3a3d03e2d345e4112a86456b1c"
server_key_iv: "c5590c2a42439daebf109d68eb32633f"
server_key_version: "encryptionkey@0"
I20260806 22:32:38.851912 2736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:38.881803 2736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:38.886010 2736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:38.886329 2736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:38.906049 2736 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43967
I20260806 22:32:38.906100 2833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43967 every 8 connection(s)
I20260806 22:32:38.907546 2736 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:38.911593 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:38.914110 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2736
I20260806 22:32:38.914316 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:38.914769 1057 external_mini_cluster.cc:1428] Setting key 981495101729c8f96fce3b00ac6f4136
I20260806 22:32:38.924639 2836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92: Bootstrap starting.
I20260806 22:32:38.928706 2836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:38.928726 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:38.916585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:57183 (local address 127.0.0.1:43967)
0806 22:32:38.917400 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:38.917411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:38.917458 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:32:38.919112 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:38.919133 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:38.919522 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:38.919863 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:38.920632 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:38.921994 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:38.924066 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:38.924768 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:38.924943 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:38.926344 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:38.926351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:38.926364 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:38.926909 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:38.926990 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:38.927025 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:38.927144 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:38.927450 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:32:38.927647 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":373,"threads_started":1}
I20260806 22:32:38.930239 2836 log.cc:824] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:38.933032 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:38.933581 2836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92: No bootstrap required, opened a new log
I20260806 22:32:38.943427 2836 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 22:32:38.943977 2836 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:38.944162 2836 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ed6b10f0cb4dcc9d6beed250c44c92, State: Initialized, Role: FOLLOWER
I20260806 22:32:38.944927 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:38.937526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:58243 (local address 127.0.0.1:43967)
0806 22:32:38.937758 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:38.937764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:38.938177 (+ 413us) server_negotiation.cc:407] Connection header received
0806 22:32:38.938362 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:38.938365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:38.938425 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:38.938507 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:38.939143 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:38.939920 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:38.942167 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:38.942597 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:38.942686 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:38.944057 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:38.944062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:38.944065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:38.944123 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:38.944198 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:38.944235 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:38.944388 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:38.944612 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 22:32:38.944667 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 22:32:38.944950 2836 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 22:32:38.945393 2836 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:38.945583 2836 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:38.945829 2836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:38.948616 2836 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 22:32:38.949222 2836 leader_election.cc:302] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3ed6b10f0cb4dcc9d6beed250c44c92; no voters:
I20260806 22:32:38.949777 2836 leader_election.cc:288] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:38.949918 2849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:38.950627 2849 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [term 1 LEADER]: Becoming Leader. State: Replica: e3ed6b10f0cb4dcc9d6beed250c44c92, State: Running, Role: LEADER
I20260806 22:32:38.951223 2849 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 22:32:38.952075 2836 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:38.954106 2850 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260806 22:32:38.954592 2850 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:38.954131 2852 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ed6b10f0cb4dcc9d6beed250c44c92. Latest consensus state: current_term: 1 leader_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ed6b10f0cb4dcc9d6beed250c44c92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260806 22:32:38.955315 2852 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:38.955690 2859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:38.961103 2859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:38.971088 2859 catalog_manager.cc:1347] Generated new cluster ID: 6925f989971f4840948b82ff73c1b40b
I20260806 22:32:38.971285 2859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:38.989216 2859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:38.990609 2859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:38.996975 2859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3ed6b10f0cb4dcc9d6beed250c44c92: Generated new TSK 0
I20260806 22:32:38.998163 2859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:39.012296 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2736
W20260806 22:32:39.013535 2829 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:57183
W20260806 22:32:39.014315 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2736: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:39.014464 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2736
W20260806 22:32:39.046926 2704 connection.cc:535] client connection to 127.0.0.1:43967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43967): Connection reset by peer (error 104)
2026-08-06T22:32:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:39 dist-test-slave-7vjv krb5kdc[2879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:39 dist-test-slave-7vjv krb5kdc[2879](info): set up 2 sockets
Aug 06 22:32:39 dist-test-slave-7vjv krb5kdc[2879](info): commencing operation
krb5kdc: starting...
W20260806 22:32:40.361790 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:40 dist-test-slave-7vjv krb5kdc[2879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:32:40.687628 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:41.029846 2898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:41.030143 2898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:41.030272 2898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:41.037334 2898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:41.037447 2898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:41.037501 2898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:41.037607 2898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:41.037670 2898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:41.047254 2898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:41.048600 2898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:41.049921 2898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:41.062363 2908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:41.062539 2905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:41.062494 2906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:41.063295 2898 server_base.cc:1034] running on GCE node
I20260806 22:32:41.063987 2898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:41.065332 2898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:41.066551 2898 hybrid_clock.cc:648] HybridClock initialized: now 1786055561066502 us; error 52 us; skew 500 ppm
I20260806 22:32:41.066865 2898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:41 dist-test-slave-7vjv krb5kdc[2879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:41.070204 2898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:32:41.071702 2898 webserver.cc:460] Webserver started at http://127.0.0.1:35719/ using document root <none> and password file <none>
I20260806 22:32:41.072433 2898 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:41.072573 2898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:41.072880 2898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:41.075802 2898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "0ee552ceef354b9faef3fe3fb82eb34f"
format_stamp: "Formatted at 2026-08-06 22:32:41 on dist-test-slave-7vjv"
server_key: "dc8344c370170dc85947bb5bcf3140f4"
server_key_iv: "ee272dacc90a0332b796b3eb68e5d92a"
server_key_version: "encryptionkey@0"
I20260806 22:32:41.076666 2898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "0ee552ceef354b9faef3fe3fb82eb34f"
format_stamp: "Formatted at 2026-08-06 22:32:41 on dist-test-slave-7vjv"
server_key: "dc8344c370170dc85947bb5bcf3140f4"
server_key_iv: "ee272dacc90a0332b796b3eb68e5d92a"
server_key_version: "encryptionkey@0"
I20260806 22:32:41.081727 2898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:41.085124 2927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:41.086356 2898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:41.086547 2898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "0ee552ceef354b9faef3fe3fb82eb34f"
format_stamp: "Formatted at 2026-08-06 22:32:41 on dist-test-slave-7vjv"
server_key: "dc8344c370170dc85947bb5bcf3140f4"
server_key_iv: "ee272dacc90a0332b796b3eb68e5d92a"
server_key_version: "encryptionkey@0"
I20260806 22:32:41.086745 2898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:41.092384 2898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:41.096300 2898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:41.096585 2898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:41.115605 2898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41447
I20260806 22:32:41.115648 2999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41447 every 8 connection(s)
I20260806 22:32:41.117139 2898 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:41.120217 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 2898
I20260806 22:32:41.120417 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:41.120900 1057 external_mini_cluster.cc:1428] Setting key f6a96ee95a3d27e2736d9171e51b6ade
I20260806 22:32:41.121166 3000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:41.129091 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:41.122634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:40465 (local address 127.0.0.1:41447)
0806 22:32:41.123424 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:41.123436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:41.123478 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:32:41.125636 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:41.125657 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:41.126021 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:41.126357 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:41.127451 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:41.127457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:41.127474 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:41.127488 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:41.127546 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:41.127573 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:41.127749 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:41.127849 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:32:41.128124 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":372,"threads_started":1}
I20260806 22:32:41.132781 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:41.134764 3000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f: Bootstrap starting.
I20260806 22:32:41.139034 3000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:41.140496 3000 log.cc:824] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:41.140722 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:41.138084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:44135 (local address 127.0.0.1:41447)
0806 22:32:41.138233 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:41.138237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:41.138674 (+ 437us) server_negotiation.cc:407] Connection header received
0806 22:32:41.138844 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:41.138847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:41.138896 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:41.138984 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:41.139949 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:41.139953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:41.139956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:41.139979 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:41.140016 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:41.140031 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:41.140187 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:41.140306 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:32:41.140354 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:32:41.143357 3000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f: No bootstrap required, opened a new log
I20260806 22:32:41.150491 3000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 22:32:41.150830 3000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:41.150938 3000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee552ceef354b9faef3fe3fb82eb34f, State: Initialized, Role: FOLLOWER
I20260806 22:32:41.151532 3000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 22:32:41.151778 3000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:41.151929 3000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:41.152122 3000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:41.154830 3000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 22:32:41.155403 3000 leader_election.cc:302] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ee552ceef354b9faef3fe3fb82eb34f; no voters:
I20260806 22:32:41.156028 3000 leader_election.cc:288] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:41.156189 3015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:41.156859 3015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [term 1 LEADER]: Becoming Leader. State: Replica: 0ee552ceef354b9faef3fe3fb82eb34f, State: Running, Role: LEADER
I20260806 22:32:41.157450 3015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 22:32:41.158221 3000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:41.160378 3017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ee552ceef354b9faef3fe3fb82eb34f. Latest consensus state: current_term: 1 leader_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260806 22:32:41.160851 3017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:41.161405 3016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee552ceef354b9faef3fe3fb82eb34f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260806 22:32:41.161749 3016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:41.165185 3030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:41.171471 3030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:41.181612 3030 catalog_manager.cc:1347] Generated new cluster ID: 33f82dd6cf444e8281b77061c4bde868
I20260806 22:32:41.181738 3030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:41.199395 3030 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:41.199844 3030 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:41.211313 3030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ee552ceef354b9faef3fe3fb82eb34f: Generated new TSK 0
I20260806 22:32:41.212216 3030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:41.228144 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2898
W20260806 22:32:41.229915 2994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:40465
W20260806 22:32:41.230641 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 2898: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:41.230793 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 2898
2026-08-06T22:32:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:41 dist-test-slave-7vjv krb5kdc[3051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:41 dist-test-slave-7vjv krb5kdc[3051](info): set up 2 sockets
Aug 06 22:32:41 dist-test-slave-7vjv krb5kdc[3051](info): commencing operation
krb5kdc: starting...
W20260806 22:32:42.602958 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:42 dist-test-slave-7vjv krb5kdc[3051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:42.953395 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37291
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:37291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:43.315775 3077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:43.316211 3077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:43.316390 3077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:43.323936 3077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:43.324151 3077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:43.324267 3077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:43.324385 3077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:43.324469 3077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:43.335632 3077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:37291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:43.337209 3077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:43.338676 3077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:43.351847 3084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:43.351886 3089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:43.351868 3085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:43.352916 3077 server_base.cc:1034] running on GCE node
I20260806 22:32:43.353782 3077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:43.355377 3077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:43.356614 3077 hybrid_clock.cc:648] HybridClock initialized: now 1786055563356570 us; error 65 us; skew 500 ppm
I20260806 22:32:43.356998 3077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:43 dist-test-slave-7vjv krb5kdc[3051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055563, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:43.360288 3077 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:43.361794 3077 webserver.cc:460] Webserver started at http://127.1.8.126:46207/ using document root <none> and password file <none>
I20260806 22:32:43.362510 3077 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:43.362671 3077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:43.362998 3077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:43.366173 3077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "4472542a5bf7493ab4763abda2870178"
format_stamp: "Formatted at 2026-08-06 22:32:43 on dist-test-slave-7vjv"
server_key: "025cdd4296d969c20ceb5c7ba1a5e506"
server_key_iv: "52fbdd91f365323c9652677613eee80d"
server_key_version: "encryptionkey@0"
I20260806 22:32:43.366981 3077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "4472542a5bf7493ab4763abda2870178"
format_stamp: "Formatted at 2026-08-06 22:32:43 on dist-test-slave-7vjv"
server_key: "025cdd4296d969c20ceb5c7ba1a5e506"
server_key_iv: "52fbdd91f365323c9652677613eee80d"
server_key_version: "encryptionkey@0"
I20260806 22:32:43.372365 3077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:32:43.375984 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:43.377317 3077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:43.377526 3077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "4472542a5bf7493ab4763abda2870178"
format_stamp: "Formatted at 2026-08-06 22:32:43 on dist-test-slave-7vjv"
server_key: "025cdd4296d969c20ceb5c7ba1a5e506"
server_key_iv: "52fbdd91f365323c9652677613eee80d"
server_key_version: "encryptionkey@0"
I20260806 22:32:43.377749 3077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:43.391278 3077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:43.395660 3077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:43.395977 3077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:43.415436 3077 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:37291
I20260806 22:32:43.415477 3193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:37291 every 8 connection(s)
I20260806 22:32:43.416963 3077 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:43.418635 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3077
I20260806 22:32:43.418820 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:43.419310 1057 external_mini_cluster.cc:1428] Setting key 2876f768bcf343e826c176518b8fcf2c
I20260806 22:32:43.421506 3194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:32:43 dist-test-slave-7vjv krb5kdc[3051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:43.436841 3194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178: Bootstrap starting.
I20260806 22:32:43.440491 3194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:43.441885 3194 log.cc:824] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:43.445638 3194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178: No bootstrap required, opened a new log
I20260806 22:32:43.445871 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:43.421267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49917 (local address 127.1.8.126:37291)
0806 22:32:43.422099 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:43.422119 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:43.422169 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:32:43.424475 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:43.424504 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:43.425046 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:43.425504 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:43.426909 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:43.429095 (+ 2186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:43.430396 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:43.431509 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:43.431703 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:43.437271 (+ 5568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:43.437280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:43.437295 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:43.437345 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:43.440531 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:43.441606 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:43.441612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:43.441618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:43.441780 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:43.442304 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:43.442307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:43.442309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:43.443100 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:43.443488 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:43.444186 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 22:32:43.444520 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":346,"threads_started":1}
I20260806 22:32:43.449865 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:43.453871 3194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } }
I20260806 22:32:43.454312 3194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:43.454445 3194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4472542a5bf7493ab4763abda2870178, State: Initialized, Role: FOLLOWER
I20260806 22:32:43.455195 3194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } }
I20260806 22:32:43.455524 3194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:43.455698 3194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:43.455896 3194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:43.458279 3194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } }
I20260806 22:32:43.458801 3194 leader_election.cc:302] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4472542a5bf7493ab4763abda2870178; no voters:
I20260806 22:32:43.459483 3194 leader_election.cc:288] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:43.459640 3208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:43.462332 3194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:43.466406 3208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [term 1 LEADER]: Becoming Leader. State: Replica: 4472542a5bf7493ab4763abda2870178, State: Running, Role: LEADER
I20260806 22:32:43.467237 3208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } }
I20260806 22:32:43.470502 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:43.454390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42919 (local address 127.1.8.126:37291)
0806 22:32:43.454587 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:43.454596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:43.455145 (+ 549us) server_negotiation.cc:407] Connection header received
0806 22:32:43.455328 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:43.455331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:43.455402 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:43.455522 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:43.457202 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:43.458003 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:43.459459 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:43.460309 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:43.461083 (+ 774us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:43.463157 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:43.463161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:43.463164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:43.463196 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:43.466444 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:43.467201 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:43.467210 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:43.467213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:43.467459 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:43.467946 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:43.467952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:43.467956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:43.468408 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:43.469693 (+ 1285us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:43.469885 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:32:43.470095 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:32:43.471036 3209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4472542a5bf7493ab4763abda2870178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } } }
I20260806 22:32:43.471524 3209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:43.472055 3210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4472542a5bf7493ab4763abda2870178. Latest consensus state: current_term: 1 leader_uuid: "4472542a5bf7493ab4763abda2870178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472542a5bf7493ab4763abda2870178" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37291 } } }
I20260806 22:32:43.472424 3210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:43.480458 3226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:43.485412 3226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:43.494273 3226 catalog_manager.cc:1347] Generated new cluster ID: 2cfcc5675900459ea978e143d5bb9a9f
I20260806 22:32:43.494418 3226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:43.509683 3226 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:43.511113 3226 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:43.523438 3226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4472542a5bf7493ab4763abda2870178: Generated new TSK 0
I20260806 22:32:43.524585 3226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:43.535815 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3077
I20260806 22:32:43.538048 3189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:43.631565 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3077
W20260806 22:32:43.661054 3049 connection.cc:535] client connection to 127.1.8.126:37291 recv error: Network error: failed to read from TLS socket (remote: 127.1.8.126:37291): Connection reset by peer (error 104)
2026-08-06T22:32:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:43 dist-test-slave-7vjv krb5kdc[3252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:43 dist-test-slave-7vjv krb5kdc[3252](info): set up 2 sockets
Aug 06 22:32:43 dist-test-slave-7vjv krb5kdc[3252](info): commencing operation
krb5kdc: starting...
W20260806 22:32:45.006500 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:45 dist-test-slave-7vjv krb5kdc[3252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:45.370851 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43645
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:38281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:43645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:45.732419 3277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:45.732729 3277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:45.732827 3277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:45.740216 3277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:45.740340 3277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:45.740388 3277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:45.740453 3277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:45.740491 3277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:45.750466 3277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:38281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:43645
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:43645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:45.751796 3277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:45.753135 3277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:45.764889 3285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:45.764986 3286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:45.765017 3291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:45.766278 3277 server_base.cc:1034] running on GCE node
I20260806 22:32:45.767097 3277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:45.768608 3277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:45.769883 3277 hybrid_clock.cc:648] HybridClock initialized: now 1786055565769823 us; error 48 us; skew 500 ppm
I20260806 22:32:45.770268 3277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:45 dist-test-slave-7vjv krb5kdc[3252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:45.773577 3277 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:45.775115 3277 webserver.cc:460] Webserver started at http://127.1.8.126:43269/ using document root <none> and password file <none>
I20260806 22:32:45.775895 3277 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:45.776059 3277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:45.776400 3277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:45.779536 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "120873946fc944938dbe74903ecf6ced"
format_stamp: "Formatted at 2026-08-06 22:32:45 on dist-test-slave-7vjv"
server_key: "149ad94e9abf33f02535b80fee32e13f"
server_key_iv: "c2f86c2512d5ed77a5616efe3da343d9"
server_key_version: "encryptionkey@0"
I20260806 22:32:45.780412 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "120873946fc944938dbe74903ecf6ced"
format_stamp: "Formatted at 2026-08-06 22:32:45 on dist-test-slave-7vjv"
server_key: "149ad94e9abf33f02535b80fee32e13f"
server_key_iv: "c2f86c2512d5ed77a5616efe3da343d9"
server_key_version: "encryptionkey@0"
I20260806 22:32:45.785734 3277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:45.789384 3301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:45.790778 3277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:32:45.790980 3277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "120873946fc944938dbe74903ecf6ced"
format_stamp: "Formatted at 2026-08-06 22:32:45 on dist-test-slave-7vjv"
server_key: "149ad94e9abf33f02535b80fee32e13f"
server_key_iv: "c2f86c2512d5ed77a5616efe3da343d9"
server_key_version: "encryptionkey@0"
I20260806 22:32:45.791201 3277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:45.809677 3277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:45.814147 3277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:45.814546 3277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:45.834780 3277 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:43645
I20260806 22:32:45.834801 3375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:43645 every 8 connection(s)
I20260806 22:32:45.836346 3277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:45.836999 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3277
I20260806 22:32:45.837217 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:45.837790 1057 external_mini_cluster.cc:1428] Setting key 3eb0f364b09519da0f1f9225c418cb15
I20260806 22:32:45.841030 3377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:45.851775 3381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:45.839696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53931 (local address 127.1.8.126:43645)
0806 22:32:45.840586 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:45.840598 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:45.840641 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:32:45.842347 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:45.842368 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:45.842744 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:45.843054 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:45.843725 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:45.845142 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:45.847072 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:45.847775 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:45.847917 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:45.849341 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:45.849349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:45.849363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:45.849918 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:45.850005 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:45.850029 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:45.850164 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:45.850478 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:32:45.850705 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":468,"threads_started":1}
I20260806 22:32:45.853788 3377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced: Bootstrap starting.
I20260806 22:32:45.856037 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:45.857506 3377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:45.858883 3377 log.cc:824] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:45.862739 3377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced: No bootstrap required, opened a new log
I20260806 22:32:45.868294 3381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:45.861632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41555 (local address 127.1.8.126:43645)
0806 22:32:45.861792 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:45.861796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:45.862302 (+ 506us) server_negotiation.cc:407] Connection header received
0806 22:32:45.862534 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:45.862538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:45.862599 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:45.862707 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:45.863270 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:45.864087 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:45.865526 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:45.866031 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:45.866198 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:45.867416 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:45.867421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:45.867424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:45.867443 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:45.867486 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:45.867502 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:45.867648 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:45.867858 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:32:45.867920 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:32:45.871347 3377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } }
I20260806 22:32:45.871795 3377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:45.871914 3377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120873946fc944938dbe74903ecf6ced, State: Initialized, Role: FOLLOWER
I20260806 22:32:45.872587 3377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } }
I20260806 22:32:45.872844 3377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:45.873004 3377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:45.873199 3377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:45.875968 3377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } }
I20260806 22:32:45.876550 3377 leader_election.cc:302] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 120873946fc944938dbe74903ecf6ced; no voters:
I20260806 22:32:45.877326 3377 leader_election.cc:288] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:45.877444 3396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:45.878223 3396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [term 1 LEADER]: Becoming Leader. State: Replica: 120873946fc944938dbe74903ecf6ced, State: Running, Role: LEADER
I20260806 22:32:45.878784 3396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } }
I20260806 22:32:45.879634 3377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:45.882076 3398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [sys.catalog]: SysCatalogTable state changed. Reason: New leader 120873946fc944938dbe74903ecf6ced. Latest consensus state: current_term: 1 leader_uuid: "120873946fc944938dbe74903ecf6ced" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } } }
I20260806 22:32:45.882552 3398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:45.882400 3397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "120873946fc944938dbe74903ecf6ced" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120873946fc944938dbe74903ecf6ced" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 43645 } } }
I20260806 22:32:45.882709 3397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:45.884752 3411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:45.889392 3411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:45.899288 3411 catalog_manager.cc:1347] Generated new cluster ID: 23f1fb480c8541a4a2e01c6f2ca7ac32
I20260806 22:32:45.899436 3411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:45.921064 3411 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:45.923156 3411 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:45.933378 3411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 120873946fc944938dbe74903ecf6ced: Generated new TSK 0
I20260806 22:32:45.934294 3411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:45.946128 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3277
W20260806 22:32:45.947903 3366 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53931
W20260806 22:32:45.948741 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3277: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:45.948915 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3277
W20260806 22:32:45.977937 3248 connection.cc:535] client connection to 127.1.8.126:43645 recv error: Network error: failed to read from TLS socket (remote: 127.1.8.126:43645): Connection reset by peer (error 104)
2026-08-06T22:32:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:46 dist-test-slave-7vjv krb5kdc[3434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:46 dist-test-slave-7vjv krb5kdc[3434](info): set up 2 sockets
Aug 06 22:32:46 dist-test-slave-7vjv krb5kdc[3434](info): commencing operation
krb5kdc: starting...
W20260806 22:32:47.294165 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.001s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:47 dist-test-slave-7vjv krb5kdc[3434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:47.639719 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34437
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:39251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:34437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:47.995520 3454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:47.995812 3454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:47.995975 3454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:48.003425 3454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:48.003561 3454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:48.003648 3454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:48.003753 3454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:48.003837 3454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:48.014223 3454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:39251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:34437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:48.015542 3454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:48.016908 3454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:48.029175 3466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:48.029345 3461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:48.029277 3463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:48.030421 3454 server_base.cc:1034] running on GCE node
I20260806 22:32:48.031270 3454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:48.032838 3454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:48.034068 3454 hybrid_clock.cc:648] HybridClock initialized: now 1786055568034030 us; error 52 us; skew 500 ppm
I20260806 22:32:48.034420 3454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:48 dist-test-slave-7vjv krb5kdc[3434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:48.037599 3454 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:48.039053 3454 webserver.cc:460] Webserver started at http://127.1.8.126:42629/ using document root <none> and password file <none>
I20260806 22:32:48.039757 3454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:48.039911 3454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:48.040297 3454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:48.043323 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "dda1ccbcc8714b4cba77a5409fc2fd65"
format_stamp: "Formatted at 2026-08-06 22:32:48 on dist-test-slave-7vjv"
server_key: "6c2b89502a378e997871d1bf8350bcdb"
server_key_iv: "6e1d35a0a2a8251be6aee8b5dcca58d7"
server_key_version: "encryptionkey@0"
I20260806 22:32:48.044086 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "dda1ccbcc8714b4cba77a5409fc2fd65"
format_stamp: "Formatted at 2026-08-06 22:32:48 on dist-test-slave-7vjv"
server_key: "6c2b89502a378e997871d1bf8350bcdb"
server_key_iv: "6e1d35a0a2a8251be6aee8b5dcca58d7"
server_key_version: "encryptionkey@0"
I20260806 22:32:48.049204 3454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:32:48.052531 3477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:48.053779 3454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 22:32:48.053952 3454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "dda1ccbcc8714b4cba77a5409fc2fd65"
format_stamp: "Formatted at 2026-08-06 22:32:48 on dist-test-slave-7vjv"
server_key: "6c2b89502a378e997871d1bf8350bcdb"
server_key_iv: "6e1d35a0a2a8251be6aee8b5dcca58d7"
server_key_version: "encryptionkey@0"
I20260806 22:32:48.054143 3454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:48.059881 3454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:48.063702 3454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:48.063982 3454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:48.082466 3454 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:34437
I20260806 22:32:48.082504 3548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:34437 every 8 connection(s)
I20260806 22:32:48.083827 3454 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:48.087266 3551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:48.092617 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3454
I20260806 22:32:48.092799 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:48.093353 1057 external_mini_cluster.cc:1428] Setting key 4601a37a001da4b3525bfb95a97a96f1
I20260806 22:32:48.098243 3551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65: Bootstrap starting.
I20260806 22:32:48.101209 3556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:48.095115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46279 (local address 127.1.8.126:34437)
0806 22:32:48.096004 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:48.096018 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:48.096060 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:32:48.097838 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:48.097860 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:48.098159 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:48.098541 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:48.099585 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:48.099591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:48.099619 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:48.099633 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:48.099691 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:48.099712 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:48.100040 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:48.100225 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:32:48.100365 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":355,"threads_started":1}
I20260806 22:32:48.102044 3551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:48.103133 3551 log.cc:824] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:48.104887 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:48.106523 3551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65: No bootstrap required, opened a new log
I20260806 22:32:48.112812 3556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:48.110290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55639 (local address 127.1.8.126:34437)
0806 22:32:48.110474 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:48.110484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:48.110907 (+ 423us) server_negotiation.cc:407] Connection header received
0806 22:32:48.111053 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:48.111058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:48.111116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:48.111217 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:48.112175 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:48.112180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:48.112186 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:48.112210 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:48.112265 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:48.112284 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:48.112412 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:48.112496 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:32:48.112546 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:32:48.115950 3551 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } }
I20260806 22:32:48.116392 3551 raft_consensus.cc:374] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:48.116515 3551 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda1ccbcc8714b4cba77a5409fc2fd65, State: Initialized, Role: FOLLOWER
I20260806 22:32:48.117148 3551 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } }
I20260806 22:32:48.117386 3551 raft_consensus.cc:388] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:48.117542 3551 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:48.117718 3551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:48.119871 3551 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } }
I20260806 22:32:48.120388 3551 leader_election.cc:302] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dda1ccbcc8714b4cba77a5409fc2fd65; no voters:
I20260806 22:32:48.120965 3551 leader_election.cc:288] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:48.121084 3572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:48.122244 3572 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [term 1 LEADER]: Becoming Leader. State: Replica: dda1ccbcc8714b4cba77a5409fc2fd65, State: Running, Role: LEADER
I20260806 22:32:48.122866 3572 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } }
I20260806 22:32:48.123059 3551 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:48.125934 3576 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda1ccbcc8714b4cba77a5409fc2fd65. Latest consensus state: current_term: 1 leader_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } } }
I20260806 22:32:48.126161 3574 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1ccbcc8714b4cba77a5409fc2fd65" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 34437 } } }
I20260806 22:32:48.126386 3576 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:48.126461 3574 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:48.128480 3588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:48.132931 3588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:48.141281 3588 catalog_manager.cc:1347] Generated new cluster ID: c6b4c661bbef4e11ac594bdf11943464
I20260806 22:32:48.141391 3588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:48.159608 3588 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:48.160015 3588 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:48.166044 3588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dda1ccbcc8714b4cba77a5409fc2fd65: Generated new TSK 0
I20260806 22:32:48.166728 3588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:48.179150 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3454
W20260806 22:32:48.180292 3541 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46279
W20260806 22:32:48.181036 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3454: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:48.181185 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3454
2026-08-06T22:32:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:48 dist-test-slave-7vjv krb5kdc[3614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:48 dist-test-slave-7vjv krb5kdc[3614](info): set up 2 sockets
Aug 06 22:32:48 dist-test-slave-7vjv krb5kdc[3614](info): commencing operation
krb5kdc: starting...
W20260806 22:32:49.583859 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:49 dist-test-slave-7vjv krb5kdc[3614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:49.921644 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38679
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:50.272303 3633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:50.272591 3633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:50.272686 3633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:50.279742 3633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:50.279842 3633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:50.279888 3633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:50.279942 3633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:50.279985 3633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:50.289707 3633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:38679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:50.290886 3633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:50.292174 3633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:50.303464 3640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:50.303503 3641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:50.303423 3643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:50.305055 3633 server_base.cc:1034] running on GCE node
I20260806 22:32:50.305895 3633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:50.307240 3633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:50.308503 3633 hybrid_clock.cc:648] HybridClock initialized: now 1786055570308448 us; error 39 us; skew 500 ppm
I20260806 22:32:50.308862 3633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:50 dist-test-slave-7vjv krb5kdc[3614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:50.311961 3633 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:50.313367 3633 webserver.cc:460] Webserver started at http://127.1.8.126:44925/ using document root <none> and password file <none>
I20260806 22:32:50.314045 3633 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:50.314188 3633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:50.314491 3633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:50.317473 3633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "6a0c702662f74d80bec1ec6dc1d7f9af"
format_stamp: "Formatted at 2026-08-06 22:32:50 on dist-test-slave-7vjv"
server_key: "ebd5d29e2cf9c2a676cb47a000750d51"
server_key_iv: "7cf5cdcd43fd4076eae08ff805266dd1"
server_key_version: "encryptionkey@0"
I20260806 22:32:50.318428 3633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "6a0c702662f74d80bec1ec6dc1d7f9af"
format_stamp: "Formatted at 2026-08-06 22:32:50 on dist-test-slave-7vjv"
server_key: "ebd5d29e2cf9c2a676cb47a000750d51"
server_key_iv: "7cf5cdcd43fd4076eae08ff805266dd1"
server_key_version: "encryptionkey@0"
I20260806 22:32:50.323429 3633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:32:50.326777 3652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:50.328016 3633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:32:50.328239 3633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "6a0c702662f74d80bec1ec6dc1d7f9af"
format_stamp: "Formatted at 2026-08-06 22:32:50 on dist-test-slave-7vjv"
server_key: "ebd5d29e2cf9c2a676cb47a000750d51"
server_key_iv: "7cf5cdcd43fd4076eae08ff805266dd1"
server_key_version: "encryptionkey@0"
I20260806 22:32:50.328429 3633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:50.361680 3633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:50.365985 3633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:50.366277 3633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:50.385195 3633 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:38679
I20260806 22:32:50.385234 3738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:38679 every 8 connection(s)
I20260806 22:32:50.386529 3633 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:50.389861 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3633
I20260806 22:32:50.390049 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:50.390290 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:50.390576 1057 external_mini_cluster.cc:1428] Setting key c1fff8b406d3e88c5ce16d8a2a5f277b
I20260806 22:32:50.405114 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af: Bootstrap starting.
Aug 06 22:32:50 dist-test-slave-7vjv krb5kdc[3614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:50.409365 3743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:50.410828 3743 log.cc:824] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:50.414494 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af: No bootstrap required, opened a new log
I20260806 22:32:50.415771 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:50.392416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60837 (local address 127.1.8.126:38679)
0806 22:32:50.393272 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:50.393288 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:50.393341 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:32:50.395478 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:50.395505 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:50.396032 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:50.396491 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:50.397782 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:50.399842 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:50.401273 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:50.402381 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:50.402603 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:50.406808 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:50.406821 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:50.406843 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:50.406900 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:50.411071 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:50.411960 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:50.411967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:50.411973 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:50.412227 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:50.412639 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:50.412644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:50.412647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:50.413333 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:50.413666 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:50.413960 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 22:32:50.414388 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":342,"threads_started":1}
I20260806 22:32:50.419625 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:50.423421 3743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } }
I20260806 22:32:50.423864 3743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:50.423990 3743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0c702662f74d80bec1ec6dc1d7f9af, State: Initialized, Role: FOLLOWER
I20260806 22:32:50.424782 3743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } }
I20260806 22:32:50.425160 3743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:50.425323 3743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:50.425506 3743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:50.427666 3743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } }
I20260806 22:32:50.428257 3743 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0c702662f74d80bec1ec6dc1d7f9af; no voters:
I20260806 22:32:50.428910 3743 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:50.429093 3756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:50.432389 3756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [term 1 LEADER]: Becoming Leader. State: Replica: 6a0c702662f74d80bec1ec6dc1d7f9af, State: Running, Role: LEADER
I20260806 22:32:50.433211 3756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } }
I20260806 22:32:50.435614 3756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0c702662f74d80bec1ec6dc1d7f9af. Latest consensus state: current_term: 1 leader_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } } }
I20260806 22:32:50.436028 3756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:50.436400 3760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0c702662f74d80bec1ec6dc1d7f9af" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 38679 } } }
I20260806 22:32:50.436842 3760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:50.438643 3743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:50.439774 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:50.424013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35919 (local address 127.1.8.126:38679)
0806 22:32:50.424207 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:50.424211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:50.424764 (+ 553us) server_negotiation.cc:407] Connection header received
0806 22:32:50.424951 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:50.424957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:50.425040 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:50.425155 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:50.426588 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:50.427279 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:50.428748 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:50.429447 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:50.429597 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:50.432020 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:50.432025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:50.432028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:50.432057 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:50.435596 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:50.436560 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:50.436567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:50.436573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:50.436726 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:50.437234 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:50.437241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:50.437248 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:50.437650 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:50.438845 (+ 1195us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:50.439019 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:32:50.439249 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:32:50.443850 3765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:50.450345 3765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:50.458369 3765 catalog_manager.cc:1347] Generated new cluster ID: f0ee4c569bab4e32bed1919b0c08ae7a
I20260806 22:32:50.458492 3765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:50.498801 3765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:50.500698 3765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:50.508601 3765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0c702662f74d80bec1ec6dc1d7f9af: Generated new TSK 0
I20260806 22:32:50.509603 3765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:50.516656 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3633
I20260806 22:32:50.517895 3733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:32:50.610713 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3633
2026-08-06T22:32:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:50 dist-test-slave-7vjv krb5kdc[3811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:50 dist-test-slave-7vjv krb5kdc[3811](info): set up 2 sockets
Aug 06 22:32:50 dist-test-slave-7vjv krb5kdc[3811](info): commencing operation
krb5kdc: starting...
W20260806 22:32:51.944340 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:52 dist-test-slave-7vjv krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:52.283422 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37223
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:37223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:52.637400 3826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:52.637698 3826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:52.637876 3826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:52.645367 3826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:52.645502 3826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:52.645630 3826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:52.645737 3826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:52.645820 3826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:52.655638 3826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:37223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:37223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:52.656827 3826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:52.658090 3826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:52.669550 3836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:52.669814 3831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:52.669592 3834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:52.670523 3826 server_base.cc:1034] running on GCE node
I20260806 22:32:52.671273 3826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:52.672628 3826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:52.673858 3826 hybrid_clock.cc:648] HybridClock initialized: now 1786055572673823 us; error 46 us; skew 500 ppm
I20260806 22:32:52.674198 3826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:52 dist-test-slave-7vjv krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055572, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:52.677424 3826 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:52.678846 3826 webserver.cc:460] Webserver started at http://127.1.8.126:36779/ using document root <none> and password file <none>
I20260806 22:32:52.679529 3826 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:52.679695 3826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:52.680014 3826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:52.682988 3826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "b3d96714c2704390a5179b38516e7384"
format_stamp: "Formatted at 2026-08-06 22:32:52 on dist-test-slave-7vjv"
server_key: "196570b6d4dc28df02df126928e24d8a"
server_key_iv: "a4a812a40d861e40619f8a2bd4bb1cc2"
server_key_version: "encryptionkey@0"
I20260806 22:32:52.683719 3826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "b3d96714c2704390a5179b38516e7384"
format_stamp: "Formatted at 2026-08-06 22:32:52 on dist-test-slave-7vjv"
server_key: "196570b6d4dc28df02df126928e24d8a"
server_key_iv: "a4a812a40d861e40619f8a2bd4bb1cc2"
server_key_version: "encryptionkey@0"
I20260806 22:32:52.688773 3826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 22:32:52.692276 3845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:52.693542 3826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 22:32:52.693732 3826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "b3d96714c2704390a5179b38516e7384"
format_stamp: "Formatted at 2026-08-06 22:32:52 on dist-test-slave-7vjv"
server_key: "196570b6d4dc28df02df126928e24d8a"
server_key_iv: "a4a812a40d861e40619f8a2bd4bb1cc2"
server_key_version: "encryptionkey@0"
I20260806 22:32:52.693918 3826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:52.720038 3826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:52.723948 3826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:52.724272 3826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:52.742698 3826 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:37223
I20260806 22:32:52.742738 3931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:37223 every 8 connection(s)
I20260806 22:32:52.744148 3826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:52.747701 3932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:52.753751 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 3826
I20260806 22:32:52.753921 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:52.754349 1057 external_mini_cluster.cc:1428] Setting key 334f5a9cfef602f528f5384302c867a0
I20260806 22:32:52.759073 3932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384: Bootstrap starting.
I20260806 22:32:52.763115 3932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:52.764187 3932 log.cc:824] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:52.766829 3932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384: No bootstrap required, opened a new log
I20260806 22:32:52.768941 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:52.756179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45913 (local address 127.1.8.126:37223)
0806 22:32:52.757008 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:52.757022 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:52.757074 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:32:52.759421 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:52.759447 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:52.759955 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:52.760401 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:52.760953 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:52.762921 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:52.764266 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:52.765260 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:52.765458 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:52.766614 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:52.766621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:52.766634 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:52.767183 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:52.767276 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:52.767303 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:52.767435 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:52.767681 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:32:52.767929 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":374,"threads_started":1}
I20260806 22:32:52.772691 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:52.775272 3932 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } }
I20260806 22:32:52.775689 3932 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:52.775807 3932 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d96714c2704390a5179b38516e7384, State: Initialized, Role: FOLLOWER
I20260806 22:32:52.776407 3932 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } }
I20260806 22:32:52.776644 3932 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:52.776798 3932 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:52.777051 3932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:52.779274 3932 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } }
I20260806 22:32:52.779775 3932 leader_election.cc:302] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3d96714c2704390a5179b38516e7384; no voters:
I20260806 22:32:52.780452 3932 leader_election.cc:288] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:52.780583 3948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:52.782356 3932 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:52.784055 3948 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [term 1 LEADER]: Becoming Leader. State: Replica: b3d96714c2704390a5179b38516e7384, State: Running, Role: LEADER
I20260806 22:32:52.784801 3948 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } }
I20260806 22:32:52.785784 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:52.777086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33469 (local address 127.1.8.126:37223)
0806 22:32:52.777311 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:52.777320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:52.777879 (+ 559us) server_negotiation.cc:407] Connection header received
0806 22:32:52.778091 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:52.778094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:52.778148 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:52.778257 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:52.778804 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:52.779508 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:52.781546 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:52.782477 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:52.782639 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:52.784406 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:52.784412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:52.784417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:52.784440 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:52.784482 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:52.784502 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:52.784601 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:52.785154 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 22:32:52.785274 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:32:52.787555 3951 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d96714c2704390a5179b38516e7384. Latest consensus state: current_term: 1 leader_uuid: "b3d96714c2704390a5179b38516e7384" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } } }
I20260806 22:32:52.787869 3951 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:52.788491 3950 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3d96714c2704390a5179b38516e7384" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d96714c2704390a5179b38516e7384" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 37223 } } }
I20260806 22:32:52.788888 3950 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:52.789768 3966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:52.796155 3966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:52.805292 3966 catalog_manager.cc:1347] Generated new cluster ID: f13fda7f33d944f080badaea43a9e31d
I20260806 22:32:52.805408 3966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:52.820948 3966 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:52.822819 3966 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:52.841876 3966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3d96714c2704390a5179b38516e7384: Generated new TSK 0
I20260806 22:32:52.842924 3966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:52.854214 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3826
W20260806 22:32:52.855499 3923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45913
W20260806 22:32:52.856292 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 3826: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:52.856498 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 3826
2026-08-06T22:32:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2253 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:52 dist-test-slave-7vjv krb5kdc[3979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:52 dist-test-slave-7vjv krb5kdc[3979](info): set up 2 sockets
Aug 06 22:32:52 dist-test-slave-7vjv krb5kdc[3979](info): commencing operation
krb5kdc: starting...
W20260806 22:32:54.201424 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:54 dist-test-slave-7vjv krb5kdc[3979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:54.527307 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33449
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:33449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:54.871776 4001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:54.872155 4001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:54.872318 4001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:54.879626 4001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:54.879789 4001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:54.879894 4001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:54.880008 4001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:54.880093 4001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:54.889993 4001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:33449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:33449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:54.891331 4001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:54.892712 4001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:54.904827 4014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:54.904932 4008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:54.904887 4012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:54.905938 4001 server_base.cc:1034] running on GCE node
I20260806 22:32:54.906790 4001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:54.908377 4001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:54.909629 4001 hybrid_clock.cc:648] HybridClock initialized: now 1786055574909593 us; error 48 us; skew 500 ppm
I20260806 22:32:54.910025 4001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:54 dist-test-slave-7vjv krb5kdc[3979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:54.917630 4001 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:54.919548 4001 webserver.cc:460] Webserver started at http://127.1.8.126:42567/ using document root <none> and password file <none>
I20260806 22:32:54.920467 4001 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:54.920640 4001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:54.921007 4001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:54.924439 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "257c651be2394001b1da36cf9b5b29a9"
format_stamp: "Formatted at 2026-08-06 22:32:54 on dist-test-slave-7vjv"
server_key: "b066e9f23edb52d15b5d8051f98b189c"
server_key_iv: "f1cb1befb879cb526f24bcd11a81f5c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:54.925303 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "257c651be2394001b1da36cf9b5b29a9"
format_stamp: "Formatted at 2026-08-06 22:32:54 on dist-test-slave-7vjv"
server_key: "b066e9f23edb52d15b5d8051f98b189c"
server_key_iv: "f1cb1befb879cb526f24bcd11a81f5c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:54.931285 4001 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 22:32:54.935159 4025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:54.936698 4001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:32:54.936902 4001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "257c651be2394001b1da36cf9b5b29a9"
format_stamp: "Formatted at 2026-08-06 22:32:54 on dist-test-slave-7vjv"
server_key: "b066e9f23edb52d15b5d8051f98b189c"
server_key_iv: "f1cb1befb879cb526f24bcd11a81f5c5"
server_key_version: "encryptionkey@0"
I20260806 22:32:54.937141 4001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:54.943403 4001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:54.948704 4001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:54.949050 4001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:54.971488 4001 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:33449
I20260806 22:32:54.971542 4102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:33449 every 8 connection(s)
I20260806 22:32:54.973047 4001 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:54.976791 4104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:54.981194 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4001
I20260806 22:32:54.981386 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:54.981814 1057 external_mini_cluster.cc:1428] Setting key 9a4cc3d814f178fb7177aa7bd3a132b6
I20260806 22:32:54.987684 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9: Bootstrap starting.
I20260806 22:32:54.989595 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:54.983557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50629 (local address 127.1.8.126:33449)
0806 22:32:54.984553 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:54.984568 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:54.984608 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:32:54.986311 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:54.986333 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:54.986655 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:54.986958 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:54.988059 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:54.988065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:54.988078 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:54.988145 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:54.988207 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:54.988233 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:54.988422 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:54.988516 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:32:54.988705 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":466,"threads_started":1}
I20260806 22:32:54.991075 4104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:54.992365 4104 log.cc:824] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:54.993418 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:54.995082 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9: No bootstrap required, opened a new log
I20260806 22:32:55.000290 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:54.997683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48557 (local address 127.1.8.126:33449)
0806 22:32:54.997834 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:54.997838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:54.998307 (+ 469us) server_negotiation.cc:407] Connection header received
0806 22:32:54.998482 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:54.998487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:54.998555 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:54.998663 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:54.999548 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:54.999554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:54.999558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:32:54.999585 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:54.999653 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:32:54.999670 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:54.999812 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:54.999906 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:32:54.999969 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:32:55.004015 4104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } }
I20260806 22:32:55.004477 4104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:32:55.004590 4104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257c651be2394001b1da36cf9b5b29a9, State: Initialized, Role: FOLLOWER
I20260806 22:32:55.005154 4104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } }
I20260806 22:32:55.005409 4104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:32:55.005584 4104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:32:55.005759 4104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:55.008503 4104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } }
I20260806 22:32:55.009034 4104 leader_election.cc:302] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 257c651be2394001b1da36cf9b5b29a9; no voters:
I20260806 22:32:55.009618 4104 leader_election.cc:288] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:32:55.009755 4122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:55.010450 4122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [term 1 LEADER]: Becoming Leader. State: Replica: 257c651be2394001b1da36cf9b5b29a9, State: Running, Role: LEADER
I20260806 22:32:55.011193 4122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } }
I20260806 22:32:55.011826 4104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:55.014829 4124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257c651be2394001b1da36cf9b5b29a9. Latest consensus state: current_term: 1 leader_uuid: "257c651be2394001b1da36cf9b5b29a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } } }
I20260806 22:32:55.015311 4124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:55.015795 4123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "257c651be2394001b1da36cf9b5b29a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257c651be2394001b1da36cf9b5b29a9" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 33449 } } }
I20260806 22:32:55.016167 4123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:55.017678 4142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:55.022771 4142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:55.031877 4142 catalog_manager.cc:1347] Generated new cluster ID: ecea68d0fb784cad89841f8e1e35d055
I20260806 22:32:55.032029 4142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:55.061822 4142 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:55.062258 4142 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:55.084030 4142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 257c651be2394001b1da36cf9b5b29a9: Generated new TSK 0
I20260806 22:32:55.085188 4142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:55.102689 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4001
W20260806 22:32:55.103796 4097 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50629
W20260806 22:32:55.104592 1057 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:32:55.104749 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4001
2026-08-06T22:32:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2243 ms)
[----------] 18 tests from AuthTokenIssuingTest (41551 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:32:55 dist-test-slave-7vjv krb5kdc[4154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:32:55 dist-test-slave-7vjv krb5kdc[4154](info): set up 2 sockets
Aug 06 22:32:55 dist-test-slave-7vjv krb5kdc[4154](info): commencing operation
krb5kdc: starting...
W20260806 22:32:56.456972 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.008s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:32:56 dist-test-slave-7vjv krb5kdc[4154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:32:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:32:56.816053 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:46187
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:32:57.165789 4175 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:32:57.166164 4175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:57.166293 4175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:57.166432 4175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:57.173588 4175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:57.173717 4175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:57.173826 4175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:57.173918 4175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:57.174008 4175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:57.183744 4175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:46187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:57.185154 4175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:57.186497 4175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:57.198258 4184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:57.198338 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:57.198314 4188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:57.199227 4175 server_base.cc:1034] running on GCE node
I20260806 22:32:57.200062 4175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:57.201507 4175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:57.202751 4175 hybrid_clock.cc:648] HybridClock initialized: now 1786055577202645 us; error 81 us; skew 500 ppm
I20260806 22:32:57.203074 4175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:57.206324 4175 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:32:57.207841 4175 webserver.cc:460] Webserver started at http://127.1.8.126:34411/ using document root <none> and password file <none>
I20260806 22:32:57.208611 4175 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:57.208758 4175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:57.209077 4175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:57.212011 4175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "7bf3e0acb7764a2f90bc2afb1317af9a"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "b8b1f95e234fba79a3dd644fbacf8050"
server_key_iv: "53ab7118a180c61e517910b23b375910"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.212852 4175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "7bf3e0acb7764a2f90bc2afb1317af9a"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "b8b1f95e234fba79a3dd644fbacf8050"
server_key_iv: "53ab7118a180c61e517910b23b375910"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.217965 4175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:32:57.221345 4197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:57.222608 4175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:32:57.222797 4175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "7bf3e0acb7764a2f90bc2afb1317af9a"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "b8b1f95e234fba79a3dd644fbacf8050"
server_key_iv: "53ab7118a180c61e517910b23b375910"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.223013 4175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:57.245616 4175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:57.250267 4175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:57.250573 4175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:57.269855 4175 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:46187
I20260806 22:32:57.269876 4280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:46187 every 8 connection(s)
I20260806 22:32:57.271261 4175 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:32:57.274991 4282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:57.281771 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4175
I20260806 22:32:57.282009 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:32:57.282955 1057 external_mini_cluster.cc:1428] Setting key 929bd3740965905389f74e6590e5aa7a
I20260806 22:32:57.282027 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:57.309535 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.286048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:46187 (local address 127.0.0.1:35362)
0806 22:32:57.287075 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.287135 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:57.290022 (+ 2887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:57.290853 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:57.290863 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:57.291733 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:57.292275 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.292304 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.293901 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.293909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.295774 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.295798 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.296790 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.296799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.297021 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.297797 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:57.297838 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:57.300919 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:57.305392 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.305403 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.305411 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.305810 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.306315 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.306318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.306320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.306463 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.307516 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:57.307523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:57.307722 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 22:32:57.307930 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 22:32:57.308446 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":470,"threads_started":1}
I20260806 22:32:57.309638 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.286048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.1.8.126:46187)
0806 22:32:57.288063 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:57.288069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:57.288177 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:32:57.290396 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:57.290401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:57.290616 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:57.290780 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:57.292506 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.293702 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.295927 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.296535 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.296669 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.301176 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:57.301184 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:57.301197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:57.301231 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:57.305245 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.305979 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.305990 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.305996 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.306205 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.306591 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.306598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.306604 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.307341 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:57.307585 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:57.308187 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 22:32:57.308657 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":1804,"spinlock_wait_cycles":15744,"thread_start_us":1575,"threads_started":1}
I20260806 22:32:57.312786 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:32:57.313995 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.313475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60312)
0806 22:32:57.313690 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.313827 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:32:57.314306 4205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
I20260806 22:32:57.316062 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.315587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60328)
0806 22:32:57.315771 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.315878 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:32:57.316301 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:45883: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
I20260806 22:32:57.365659 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } attempt: 1
I20260806 22:32:57.366950 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.366482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60338)
0806 22:32:57.366698 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.366793 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 22:32:57.367143 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:45883
--webserver_interface=127.1.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:32:57.367980 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.367613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60340)
0806 22:32:57.367761 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.367838 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:32:57.368224 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:45883: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
I20260806 22:32:57.436457 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } attempt: 2
I20260806 22:32:57.437794 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.437252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60348)
0806 22:32:57.437547 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.437642 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 22:32:57.438786 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.438432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60360)
0806 22:32:57.438576 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.438661 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:32:57.438963 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:45883: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
I20260806 22:32:57.530254 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } attempt: 3
I20260806 22:32:57.531510 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.531021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60366)
0806 22:32:57.531236 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.531351 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 22:32:57.532673 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.532294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60374)
0806 22:32:57.532458 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.532527 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:32:57.532897 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:45883: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
I20260806 22:32:57.676198 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } attempt: 4
I20260806 22:32:57.677523 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.676945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60390)
0806 22:32:57.677264 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.677387 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260806 22:32:57.678543 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.678179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60398)
0806 22:32:57.678331 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.678422 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 22:32:57.678731 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:45883: Network error: Client connection negotiation failed: client connection to 127.1.8.125:45883: connect: Connection refused (error 111)
W20260806 22:32:57.722009 4292 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:32:57.722393 4292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:57.722538 4292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:57.722681 4292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:57.729955 4292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:57.730103 4292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:57.730219 4292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:57.730302 4292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:57.730412 4292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:57.740216 4292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:57.741568 4292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:57.742900 4292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:57.754834 4298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:57.754742 4299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:57.755434 4301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:57.755570 4292 server_base.cc:1034] running on GCE node
I20260806 22:32:57.756455 4292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:57.757874 4292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:57.759097 4292 hybrid_clock.cc:648] HybridClock initialized: now 1786055577759021 us; error 55 us; skew 500 ppm
I20260806 22:32:57.759433 4292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:57.762651 4292 init.cc:377] Logged in from keytab as kudu/127.1.8.125@KRBTEST.COM (short username kudu)
I20260806 22:32:57.764245 4292 webserver.cc:460] Webserver started at http://127.1.8.125:46631/ using document root <none> and password file <none>
I20260806 22:32:57.764950 4292 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:57.765079 4292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:57.765399 4292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:57.768512 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data/instance:
uuid: "f97690a33bfe4c379a0a111d5acdc5fe"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "2b5002edf7bbca50f7ed67fdd074047f"
server_key_iv: "42ae5cb94161c631f18e1bcf2d39d4c6"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.769311 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal/instance:
uuid: "f97690a33bfe4c379a0a111d5acdc5fe"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "2b5002edf7bbca50f7ed67fdd074047f"
server_key_iv: "42ae5cb94161c631f18e1bcf2d39d4c6"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.774580 4292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:32:57.778060 4315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:57.779405 4292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:32:57.779589 4292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "f97690a33bfe4c379a0a111d5acdc5fe"
format_stamp: "Formatted at 2026-08-06 22:32:57 on dist-test-slave-7vjv"
server_key: "2b5002edf7bbca50f7ed67fdd074047f"
server_key_iv: "42ae5cb94161c631f18e1bcf2d39d4c6"
server_key_version: "encryptionkey@0"
I20260806 22:32:57.779820 4292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:57.802702 4292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:57.806869 4292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:57.807159 4292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:57.826534 4292 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.125:45883
I20260806 22:32:57.826658 4390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.125:45883 every 8 connection(s)
I20260806 22:32:57.827939 4292 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
I20260806 22:32:57.831863 4391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:57.835021 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4292
I20260806 22:32:57.835242 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-1/wal/instance
I20260806 22:32:57.835912 1057 external_mini_cluster.cc:1428] Setting key 017a28c7dd91e07addc74dd7fa5e2e55
I20260806 22:32:57.841673 4391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:57.866743 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.845264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.1.8.126:46187)
0806 22:32:57.845869 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:57.845876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:57.846934 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 22:32:57.850044 (+ 3110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:57.850052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:57.850140 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:57.850252 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:57.852514 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.853520 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.855677 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.856206 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.856322 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.860778 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:57.860784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:57.860788 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:57.860830 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:57.863548 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.864381 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.864387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.864392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.864519 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.864964 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.864969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.864974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.865259 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:57.865424 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:57.866132 (+ 708us) server_negotiation.cc:299] Negotiation successful
0806 22:32:57.866357 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":318,"threads_started":1}
I20260806 22:32:57.867746 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.845796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:46187 (local address 127.0.0.1:35378)
0806 22:32:57.846722 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.846775 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:57.849697 (+ 2922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:57.850539 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:57.850553 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:57.851388 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:57.852278 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.852308 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.853704 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.853709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.855501 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.855520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.856453 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.856462 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.856801 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.857613 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:57.857654 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:57.860527 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:57.863698 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.863707 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.863724 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.864250 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.864637 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.864642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.864646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.864836 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.865404 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:57.865413 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:57.865764 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 22:32:57.866038 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 22:32:57.866572 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":350,"threads_started":1}
I20260806 22:32:57.869920 4391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.124@KRBTEST.COM" created.
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:32:57.886195 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.870690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60406)
0806 22:32:57.870986 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.871006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:57.871134 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:57.872211 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:57.872217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:57.872767 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:57.872993 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.873007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.874565 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.874572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.875504 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.875523 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.876613 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.876620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.876818 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.877493 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:57.877512 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:57.879783 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:57.883288 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.883296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.883302 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.883720 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.884223 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.884229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.884232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.884386 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.885103 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:57.885110 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:57.885289 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:32:57.885445 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:32:57.885752 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 22:32:57.886318 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.870964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.1.8.125:45883)
0806 22:32:57.871738 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:57.871744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:57.871773 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:32:57.871818 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:57.871824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:57.872008 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:57.872239 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:57.873197 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.874398 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.875709 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.876437 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.876595 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.880015 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:57.880028 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:57.880042 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:57.880077 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:57.883137 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.883897 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.883906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.883910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.884073 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.884510 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.884515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.884519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.884922 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:57.885191 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:57.885607 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 22:32:57.885886 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":617,"thread_start_us":446,"threads_started":1}
I20260806 22:32:57.888924 4391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:32:57.890195 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.889617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35940)
0806 22:32:57.889836 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.889986 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 22:32:57.890478 4322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:57.892331 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.891837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35954)
0806 22:32:57.892052 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.892181 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 22:32:57.892536 4391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
I20260806 22:32:57.919553 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:43607
--webserver_interface=127.1.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:32:57.941845 4391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 1
I20260806 22:32:57.943210 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.942571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35970)
0806 22:32:57.942845 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.942990 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 22:32:57.944409 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.943929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35976)
0806 22:32:57.944130 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.944250 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:32:57.944643 4391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:57.977989 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } attempt: 5
Aug 06 22:32:57 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:32:57.993402 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.978885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.1.8.125:45883)
0806 22:32:57.979134 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:57.979142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:57.979241 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:32:57.979535 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:57.979545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:57.979652 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:57.979769 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:57.980658 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.981766 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.982959 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.983787 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.983955 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.987892 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:57.987898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:57.987901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:57.987939 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:57.990662 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.991505 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.991509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.991511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.991628 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:57.992056 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:57.992059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:57.992061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:57.992417 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:57.992533 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:57.992994 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 22:32:57.993176 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:32:57.993623 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.978855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60410)
0806 22:32:57.979115 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.979140 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:57.979344 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:57.979815 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:57.979819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:57.980302 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:57.980492 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.980501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.981920 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.981924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.982798 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:57.982809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:57.983950 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:57.983954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:57.984286 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:57.985087 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:57.985107 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:57.987652 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:57.990825 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.990830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.990834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.991387 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.991738 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:57.991743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:57.991747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:57.991929 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:57.992540 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:57.992544 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:57.992735 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 22:32:57.992901 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 22:32:57.993253 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:32:57.994823 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:32:57.995898 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.995514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35978)
0806 22:32:57.995679 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.995759 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:32:57.996872 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:57.996544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:35994)
0806 22:32:57.996699 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:57.996772 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:32:57.997098 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.012880 4391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 2
I20260806 22:32:58.014235 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.013640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36010)
0806 22:32:58.013888 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.014013 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:32:58.015295 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.014895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36022)
0806 22:32:58.015052 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.015135 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:32:58.015532 4391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.048334 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 1
I20260806 22:32:58.049628 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.049125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36030)
0806 22:32:58.049337 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.049432 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 22:32:58.050778 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.050395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36038)
0806 22:32:58.050538 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.050640 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:32:58.051026 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.106801 4391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 3
I20260806 22:32:58.108273 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.107647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36052)
0806 22:32:58.107903 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.108031 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 22:32:58.109436 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.108951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36058)
0806 22:32:58.109185 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.109295 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260806 22:32:58.109681 4391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.119323 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 2
I20260806 22:32:58.120590 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.120115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36072)
0806 22:32:58.120334 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.120433 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:32:58.121632 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.121271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36082)
0806 22:32:58.121431 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.121530 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:32:58.121845 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.228085 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 3
I20260806 22:32:58.229228 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.228752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36098)
0806 22:32:58.228954 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.229054 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:32:58.230253 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.229870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36100)
0806 22:32:58.230057 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.230146 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 22:32:58.230464 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
I20260806 22:32:58.252921 4391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 4
I20260806 22:32:58.254346 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.253692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36112)
0806 22:32:58.254003 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.254131 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:32:58.255461 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.255048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36122)
0806 22:32:58.255235 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.255321 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:32:58.255692 4391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:43607: Network error: Client connection negotiation failed: client connection to 127.1.8.124:43607: connect: Connection refused (error 111)
W20260806 22:32:58.285393 4404 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:32:58.285766 4404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:32:58.285892 4404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:32:58.286034 4404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:32:58.293340 4404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:32:58.293434 4404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:32:58.293494 4404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:32:58.293579 4404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:32:58.293675 4404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:32:58.303560 4404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:46187,127.1.8.125:45883,127.1.8.124:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:32:58.304991 4404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:32:58.306308 4404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:32:58.318325 4413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:58.318374 4422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:32:58.318452 4415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:32:58.319272 4404 server_base.cc:1034] running on GCE node
I20260806 22:32:58.320019 4404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:32:58.321353 4404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:32:58.322715 4404 hybrid_clock.cc:648] HybridClock initialized: now 1786055578322632 us; error 49 us; skew 500 ppm
I20260806 22:32:58.323071 4404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:32:58.326371 4404 init.cc:377] Logged in from keytab as kudu/127.1.8.124@KRBTEST.COM (short username kudu)
I20260806 22:32:58.327911 4404 webserver.cc:460] Webserver started at http://127.1.8.124:44879/ using document root <none> and password file <none>
I20260806 22:32:58.328665 4404 fs_manager.cc:359] Metadata directory not provided
I20260806 22:32:58.328804 4404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:32:58.329150 4404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:32:58.332279 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data/instance:
uuid: "673e0e92930c4bd290e80e57a4f73248"
format_stamp: "Formatted at 2026-08-06 22:32:58 on dist-test-slave-7vjv"
server_key: "c75009a71009cbba0c84cfee213a9ab8"
server_key_iv: "1ed71864aa97d1f2e63834c029ef852d"
server_key_version: "encryptionkey@0"
I20260806 22:32:58.333068 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal/instance:
uuid: "673e0e92930c4bd290e80e57a4f73248"
format_stamp: "Formatted at 2026-08-06 22:32:58 on dist-test-slave-7vjv"
server_key: "c75009a71009cbba0c84cfee213a9ab8"
server_key_iv: "1ed71864aa97d1f2e63834c029ef852d"
server_key_version: "encryptionkey@0"
I20260806 22:32:58.339095 4404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:32:58.343760 4439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:32:58.345152 4404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 22:32:58.345340 4404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "673e0e92930c4bd290e80e57a4f73248"
format_stamp: "Formatted at 2026-08-06 22:32:58 on dist-test-slave-7vjv"
server_key: "c75009a71009cbba0c84cfee213a9ab8"
server_key_iv: "1ed71864aa97d1f2e63834c029ef852d"
server_key_version: "encryptionkey@0"
I20260806 22:32:58.345600 4404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:32:58.363044 4404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:32:58.367650 4404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:32:58.367913 4404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:32:58.387858 4404 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.124:43607
I20260806 22:32:58.388012 4517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.124:43607 every 8 connection(s)
I20260806 22:32:58.390004 4404 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
I20260806 22:32:58.396711 4518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:32:58.397790 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4404
I20260806 22:32:58.398082 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055191034056-1057-0/minicluster-data/master-2/wal/instance
I20260806 22:32:58.398695 1057 external_mini_cluster.cc:1428] Setting key ed7a238d3a23e19026aee5c40b10b092
I20260806 22:32:58.407706 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 4
I20260806 22:32:58.408675 4518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:58.428210 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.408478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36138)
0806 22:32:58.408790 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.408812 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:58.408931 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:58.409927 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:58.409934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:58.410464 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:58.410631 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.410642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.411876 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.411882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.413714 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.413744 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.415652 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.415658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.415854 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.416548 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:58.416583 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:58.418858 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:58.424572 (+ 5714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.424578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.424582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.425164 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.425676 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.425681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.425685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.425853 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.426858 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:58.426862 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:58.427041 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:32:58.427277 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 22:32:58.427614 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260806 22:32:58.430275 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.408686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.1.8.124:43607)
0806 22:32:58.409553 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.409559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.409587 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:32:58.409652 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.409657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.409728 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.409849 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.410835 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.411710 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.413987 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.415440 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.416340 (+ 900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.419145 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.419154 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.419165 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.419205 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.424390 (+ 5185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.425336 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.425343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.425347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.425518 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.425997 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.426004 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.426006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.426660 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.426896 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.428296 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.428922 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":237,"server-negotiator.queue_time_us":695,"thread_start_us":486,"threads_started":1}
I20260806 22:32:58.430299 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.401142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.1.8.124:43607)
0806 22:32:58.402053 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.402070 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.402129 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:32:58.405120 (+ 2991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.405154 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.405765 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.406290 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.408089 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.410117 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.412034 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.413309 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.413549 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.420159 (+ 6610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.420166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.420174 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.420213 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.424456 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.425217 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.425232 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.425238 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.425468 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.425936 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.425940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.425943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.426628 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.426928 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.428265 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.428822 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":623,"thread_start_us":373,"threads_started":1}
I20260806 22:32:58.433167 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.412623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:46187 (local address 127.0.0.1:35394)
0806 22:32:58.415906 (+ 3283us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.415959 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:58.416244 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:58.416992 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:58.417001 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:58.417946 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:58.418190 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.418199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.419891 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.419899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.421473 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.421494 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.422559 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.422568 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.422756 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.423367 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:58.423384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:58.426170 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:58.430010 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.430031 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.430039 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.430663 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.431127 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.431132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.431135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.431310 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.431991 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:58.431999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:58.432199 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:32:58.432400 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 22:32:58.432706 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3013,"thread_start_us":2849,"threads_started":1}
I20260806 22:32:58.433218 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.412563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.1.8.126:46187)
0806 22:32:58.415282 (+ 2719us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.415289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.416064 (+ 775us) server_negotiation.cc:407] Connection header received
0806 22:32:58.416420 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.416427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.416543 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.416646 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.418384 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.419669 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.421691 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.422326 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.422466 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.426393 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.426399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.426403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.426439 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.429869 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.430797 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.430804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.430808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.430939 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.431458 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.431473 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.431476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.431856 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.431978 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.432541 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.432780 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2526,"thread_start_us":2368,"threads_started":1}
I20260806 22:32:58.435035 4518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:32:58.438076 4282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: Bootstrap starting.
I20260806 22:32:58.442492 4282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: Neither blocks nor log segments found. Creating new log.
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:32:58.443782 4282 log.cc:824] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:32:58.447971 4282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: No bootstrap required, opened a new log
I20260806 22:32:58.451443 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.435747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.1.8.125:45883)
0806 22:32:58.436295 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.436299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.436316 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:32:58.436368 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.436371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.436431 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.436519 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.437443 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.438591 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.439784 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.440326 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.440468 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.443986 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.443993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.443997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.444040 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.448028 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.448999 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.449005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.449008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.449182 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.449665 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.449670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.449674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.450174 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.450326 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.450824 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.451058 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438}
I20260806 22:32:58.452428 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.435624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:45883 (local address 127.0.0.1:60426)
0806 22:32:58.435853 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.435871 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:58.435962 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:58.436590 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:58.436596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:58.437090 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:58.437259 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.437271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.438742 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.438753 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.439628 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.439639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.440493 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.440499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.440673 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.441233 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:58.441249 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:58.443763 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:58.448241 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.448245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.448249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.448821 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.449328 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.449333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.449336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.449530 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.450333 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:58.450339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:58.450520 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:32:58.451716 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0806 22:32:58.452015 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 22:32:58.453635 4518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:32:58.454049 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.436727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.1.8.126:46187)
0806 22:32:58.436902 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.436906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.436920 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:32:58.436957 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.436967 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.437037 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.437144 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.438663 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.439724 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.441588 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.442271 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.442429 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.447115 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.447121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.447124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.447155 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.450553 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.451195 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.451201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.451204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.451335 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.451822 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.451825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.451827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.452192 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.452513 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.453489 (+ 976us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.453716 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:32:58.459383 4282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.459821 4282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf3e0acb7764a2f90bc2afb1317af9a, State: Initialized, Role: FOLLOWER
I20260806 22:32:58.460399 4282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.461516 4530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.462077 4530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:32:58.462862 4282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:32:58.469918 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.454661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.1.8.124:43607)
0806 22:32:58.454805 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.454812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.454832 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:32:58.454882 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.454887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.454952 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.455045 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.456033 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.457126 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.458802 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.459396 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.459518 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.463493 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.463501 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.463506 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.463558 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.466743 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.467565 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.467572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.467576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.467714 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.468123 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.468131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.468137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.468598 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.468755 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.469254 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.469617 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":45}
I20260806 22:32:58.469892 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.454369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36146)
0806 22:32:58.454565 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.454585 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:58.454695 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:58.455200 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:58.455206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:58.455683 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:58.455874 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.455885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.457276 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.457284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.458652 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.458666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.459747 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.459754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.459972 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.460607 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:58.460629 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:58.463285 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:58.466885 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.466892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.466898 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.467462 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.467825 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.467833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.467837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.467971 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.468813 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:58.468819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:58.468995 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:32:58.469152 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:32:58.469474 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":1}
I20260806 22:32:58.474992 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.455818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.1.8.125:45883)
0806 22:32:58.455994 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.456000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.456271 (+ 271us) server_negotiation.cc:407] Connection header received
0806 22:32:58.456415 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.456421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.456498 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.456593 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.457964 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.458652 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.459894 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.460612 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.460729 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.466184 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.466194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.466202 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.466253 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.471127 (+ 4874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.472005 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.472011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.472015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.472224 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.472679 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.472684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.472687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.473045 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.473316 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.474402 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.474619 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:32:58.476054 4518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: Bootstrap starting.
I20260806 22:32:58.480262 4518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:58.481617 4518 log.cc:824] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: Log is configured to *not* fsync() on all Append() calls
I20260806 22:32:58.485034 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:32:58.476481 4175 master_runner.cc:186] Error getting master registration for 127.1.8.125:45883: OK, instance_id { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" instance_seqno: 1786055577821063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:32:58.485386 4175 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:32:58.488113 4238 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35406
I20260806 22:32:58.488391 4518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: No bootstrap required, opened a new log
I20260806 22:32:58.488564 4238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:32:58.488765 4238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:32:58.490262 4562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:32:58.490373 4562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:32:58.494063 4238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.496301 4238 leader_election.cc:288] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [CANDIDATE]: Term 1 election: Requested vote from peers f97690a33bfe4c379a0a111d5acdc5fe (127.1.8.125:45883), 673e0e92930c4bd290e80e57a4f73248 (127.1.8.124:43607)
I20260806 22:32:58.499650 4518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.500260 4518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673e0e92930c4bd290e80e57a4f73248, State: Initialized, Role: FOLLOWER
I20260806 22:32:58.500890 4518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
W20260806 22:32:58.503654 4207 tablet.cc:2369] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:58.506397 4518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:58.507067 4484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "673e0e92930c4bd290e80e57a4f73248"
I20260806 22:32:58.507720 4484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:58.520311 4484 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bf3e0acb7764a2f90bc2afb1317af9a in term 1.
I20260806 22:32:58.521710 4204 leader_election.cc:302] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673e0e92930c4bd290e80e57a4f73248, 7bf3e0acb7764a2f90bc2afb1317af9a; no voters:
I20260806 22:32:58.522349 4530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:32:58.523106 4578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.524022 4578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.538589 4530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [term 1 LEADER]: Becoming Leader. State: Replica: 7bf3e0acb7764a2f90bc2afb1317af9a, State: Running, Role: LEADER
I20260806 22:32:58.539398 4530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.553196 4582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf3e0acb7764a2f90bc2afb1317af9a. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.553763 4582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:58.556355 4391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } attempt: 5
Aug 06 22:32:58 dist-test-slave-7vjv krb5kdc[4154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:32:58.575111 4598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:32:58.578747 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.557097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:43607 (local address 127.0.0.1:36166)
0806 22:32:58.557535 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 22:32:58.557556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:32:58.557688 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:32:58.558749 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:32:58.558757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:32:58.559422 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:32:58.559701 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.559718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.561390 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.561396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.562894 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:32:58.562912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.564076 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.564084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:32:58.564334 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.564989 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:32:58.565013 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:32:58.568166 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:32:58.575636 (+ 7470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.575642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.575646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.576233 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.576711 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:32:58.576716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:32:58.576720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:32:58.576900 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:32:58.577635 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:32:58.577643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:32:58.577820 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 22:32:58.577963 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 22:32:58.578285 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297}
I20260806 22:32:58.579952 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.557949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.1.8.124:43607)
0806 22:32:58.558194 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.558203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.558226 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:32:58.558291 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.558299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.558390 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.558502 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.559919 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.561197 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.563092 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.563752 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.563882 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.568382 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.568389 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.568393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.568435 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.575497 (+ 7062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.576390 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.576396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.576401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.576565 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.577032 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.577039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.577042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.577410 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.577568 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.579347 (+ 1779us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.579618 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:32:58.582508 4598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:32:58.587544 4391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: Bootstrap starting.
I20260806 22:32:58.596952 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.528531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.1.8.124:43607)
0806 22:32:58.528752 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.528759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.537593 (+ 8834us) server_negotiation.cc:407] Connection header received
0806 22:32:58.537939 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.537946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.538080 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.538182 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.542531 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.544147 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.549524 (+ 5377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.550273 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.550377 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.570511 (+ 20134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.570521 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.570528 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.570579 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.575108 (+ 4529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.579424 (+ 4316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.579433 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.579438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.579639 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.580262 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.587726 (+ 7464us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.587748 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.592353 (+ 4605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.592724 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.596043 (+ 3319us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.596438 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:32:58.601696 4484 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [term 1 FOLLOWER]: Refusing update from remote peer 7bf3e0acb7764a2f90bc2afb1317af9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:58.602646 4582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:58.614952 4391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: Neither blocks nor log segments found. Creating new log.
I20260806 22:32:58.617070 4391 log.cc:824] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: Log is configured to *not* fsync() on all Append() calls
W20260806 22:32:58.619910 4446 tablet.cc:2369] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:32:58.629848 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.530368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.1.8.126:46187)
0806 22:32:58.530606 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.530615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.551839 (+ 21224us) server_negotiation.cc:407] Connection header received
0806 22:32:58.560131 (+ 8292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.560140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.560251 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.560349 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.566476 (+ 6127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.567899 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.593473 (+ 25574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.594523 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.594668 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.620152 (+ 25484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.620163 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.620169 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.620238 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.625153 (+ 4915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.626404 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.626414 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.626420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.626630 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.627278 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.627286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.627292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.627816 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.628202 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.628935 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.629271 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:32:58.634351 4578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf3e0acb7764a2f90bc2afb1317af9a. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.634773 4578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.641487 4602 mvcc.cc:205] Tried to move back new op lower bound from 7315683649927217152 to 7315683649704550400. Current Snapshot: MvccSnapshot[applied={T|T < 7315683649927217152}]
I20260806 22:32:58.642630 4582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.642915 4582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:58.647276 4391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: No bootstrap required, opened a new log
I20260806 22:32:58.647490 4598 catalog_manager.cc:1347] Generated new cluster ID: 19cd1a4eaa17431aadddbdeff6a9fae5
I20260806 22:32:58.647600 4598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:32:58.647742 4603 mvcc.cc:205] Tried to move back new op lower bound from 7315683649927217152 to 7315683649704550400. Current Snapshot: MvccSnapshot[applied={T|T < 7315683649927217152}]
I20260806 22:32:58.664260 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:32:58.549851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.1.8.125:45883)
0806 22:32:58.550079 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.550086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.602267 (+ 52181us) server_negotiation.cc:407] Connection header received
0806 22:32:58.608288 (+ 6021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.608298 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.608414 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.608508 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.610496 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.611930 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.631426 (+ 19496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:32:58.632218 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:32:58.632382 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:32:58.650774 (+ 18392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:32:58.650785 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:32:58.650791 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:32:58.650841 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:32:58.655421 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.657462 (+ 2041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.657471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.657477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.657964 (+ 487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:32:58.659226 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:32:58.659234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:32:58.659240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:32:58.660138 (+ 898us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:32:58.660695 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0806 22:32:58.662164 (+ 1469us) server_negotiation.cc:299] Negotiation successful
0806 22:32:58.662982 (+ 818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:32:58.674544 4391 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.675004 4391 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97690a33bfe4c379a0a111d5acdc5fe, State: Initialized, Role: FOLLOWER
I20260806 22:32:58.677054 4391 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } }
I20260806 22:32:58.679838 4578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.680124 4578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.681555 4622 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:32:58.689421 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.697818 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.697916 4361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:32:58.697021 4362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f97690a33bfe4c379a0a111d5acdc5fe"
I20260806 22:32:58.692833 4391 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:32:58.702394 4361 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [term 1 FOLLOWER]: Refusing update from remote peer 7bf3e0acb7764a2f90bc2afb1317af9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:32:58.706339 4598 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:32:58.707886 4362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bf3e0acb7764a2f90bc2afb1317af9a in term 1.
I20260806 22:32:58.690872 4404 master_runner.cc:186] Error getting master registration for 127.1.8.125:45883: OK, instance_id { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" instance_seqno: 1786055577821063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:32:58.709733 4404 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:32:58.711025 4598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:32:58.711544 4582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:32:58.730263 4598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a: Generated new TSK 0
I20260806 22:32:58.731451 4598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:32:58.802805 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf3e0acb7764a2f90bc2afb1317af9a. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.803210 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.803614 4631 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.804023 4631 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:32:58.805727 4582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bf3e0acb7764a2f90bc2afb1317af9a" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 46187 } } peers { permanent_uuid: "f97690a33bfe4c379a0a111d5acdc5fe" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 45883 } } peers { permanent_uuid: "673e0e92930c4bd290e80e57a4f73248" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 43607 } } }
I20260806 22:32:58.806092 4582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf3e0acb7764a2f90bc2afb1317af9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:32:58.821611 4644 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 22:32:58.869268 1057 mini_kdc.cc:348] Time spent kdestroy: real 0.118s user 0.007s sys 0.033s
W20260806 22:32:58.894002 4523 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.890530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.1.8.124:43607)
0806 22:32:58.890762 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.890768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.891363 (+ 595us) server_negotiation.cc:407] Connection header received
0806 22:32:58.891727 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.891737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.891854 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.891950 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.893616 (+ 1666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36178: BlockingRecv error: recv got EOF from 127.0.0.1:36178 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 22:32:58.896862 4525 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.889841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35420 (local address 127.1.8.126:46187)
0806 22:32:58.890045 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.890050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.894789 (+ 4739us) server_negotiation.cc:407] Connection header received
0806 22:32:58.895038 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.895046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.895161 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.895255 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.896504 (+ 1249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35420: BlockingRecv error: recv got EOF from 127.0.0.1:35420 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 22:32:58.899907 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.898014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.1.8.125:45883)
0806 22:32:58.898216 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.898223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.898371 (+ 148us) server_negotiation.cc:407] Connection header received
0806 22:32:58.898572 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.898581 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.898663 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.898773 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.899676 (+ 903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60460: BlockingRecv error: recv got EOF from 127.0.0.1:60460 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 22:32:58.913025 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.904635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.1.8.125:45883)
0806 22:32:58.904832 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.904841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.911438 (+ 6597us) server_negotiation.cc:407] Connection header received
0806 22:32:58.911711 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.911716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.911828 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.911945 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.912736 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60474: BlockingRecv error: recv got EOF from 127.0.0.1:60474 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 22:32:58.931178 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.928821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.1.8.125:45883)
0806 22:32:58.929002 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.929010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.929703 (+ 693us) server_negotiation.cc:407] Connection header received
0806 22:32:58.929934 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.929945 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.930022 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.930146 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.930933 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60480: BlockingRecv error: recv got EOF from 127.0.0.1:60480 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:32:58.938325 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.936041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.1.8.125:45883)
0806 22:32:58.936232 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.936237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.936911 (+ 674us) server_negotiation.cc:407] Connection header received
0806 22:32:58.937145 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.937150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.937253 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.937361 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.938099 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60492: BlockingRecv error: recv got EOF from 127.0.0.1:60492 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 22:32:58.947784 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.945587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.1.8.125:45883)
0806 22:32:58.945810 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.945820 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.946359 (+ 539us) server_negotiation.cc:407] Connection header received
0806 22:32:58.946580 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.946585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.946670 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.946779 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.947520 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60498: BlockingRecv error: recv got EOF from 127.0.0.1:60498 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 22:32:58.960979 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.958710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.1.8.125:45883)
0806 22:32:58.958889 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.958894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.959500 (+ 606us) server_negotiation.cc:407] Connection header received
0806 22:32:58.959741 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.959747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.959834 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.959945 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.960731 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60512: BlockingRecv error: recv got EOF from 127.0.0.1:60512 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 22:32:58.982542 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:58.980205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.1.8.125:45883)
0806 22:32:58.980435 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:58.980445 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:58.981000 (+ 555us) server_negotiation.cc:407] Connection header received
0806 22:32:58.981259 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:58.981264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:58.981363 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:58.981479 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:58.982252 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60516: BlockingRecv error: recv got EOF from 127.0.0.1:60516 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 22:32:59.020381 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:59.017997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.1.8.125:45883)
0806 22:32:59.018254 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:59.018263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:59.018815 (+ 552us) server_negotiation.cc:407] Connection header received
0806 22:32:59.019050 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:59.019055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:59.019134 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:59.019248 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:59.020041 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60524: BlockingRecv error: recv got EOF from 127.0.0.1:60524 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 22:32:59.090476 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:59.088192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.1.8.125:45883)
0806 22:32:59.088371 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:59.088376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:59.089021 (+ 645us) server_negotiation.cc:407] Connection header received
0806 22:32:59.089267 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:59.089273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:59.089364 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:59.089475 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:59.090240 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60532: BlockingRecv error: recv got EOF from 127.0.0.1:60532 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:32:59.224877 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:59.222728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.1.8.125:45883)
0806 22:32:59.223002 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:59.223009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:59.223305 (+ 296us) server_negotiation.cc:407] Connection header received
0806 22:32:59.223543 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:59.223549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:59.223639 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:59.223757 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:59.224618 (+ 861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60540: BlockingRecv error: recv got EOF from 127.0.0.1:60540 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 22:32:59.486982 4398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:32:59.485141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.1.8.125:45883)
0806 22:32:59.485371 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:32:59.485377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:32:59.485556 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:32:59.485745 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:32:59.485750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:32:59.485827 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:32:59.485933 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:32:59.486739 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59474: BlockingRecv error: recv got EOF from 127.0.0.1:59474 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:32:59.686982 4622 catalog_manager.cc:1259] Loaded cluster ID: 19cd1a4eaa17431aadddbdeff6a9fae5
I20260806 22:32:59.687100 4622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: loading cluster ID for follower catalog manager: success
I20260806 22:32:59.689790 4622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: acquiring CA information for follower catalog manager: success
I20260806 22:32:59.691751 4622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 673e0e92930c4bd290e80e57a4f73248: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:32:59.826858 4644 catalog_manager.cc:1259] Loaded cluster ID: 19cd1a4eaa17431aadddbdeff6a9fae5
I20260806 22:32:59.827023 4644 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: loading cluster ID for follower catalog manager: success
I20260806 22:32:59.830969 4644 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: acquiring CA information for follower catalog manager: success
I20260806 22:32:59.833532 4644 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f97690a33bfe4c379a0a111d5acdc5fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:33:00.025702 4757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:00.004392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.1.8.125:45883)
0806 22:33:00.017078 (+ 12686us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:00.017085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:00.017895 (+ 810us) server_negotiation.cc:407] Connection header received
0806 22:33:00.018123 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:00.018129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:00.018347 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:00.018505 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:00.019143 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.020549 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.022545 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.023252 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.023386 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:00.024063 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:00.024950 (+ 887us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:00.025116 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:00.025253 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:33:00.025347 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12510,"thread_start_us":12705,"threads_started":1}
I20260806 22:33:00.042798 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:00.033669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.1.8.124:43607)
0806 22:33:00.034614 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:00.034624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:00.034645 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:33:00.034747 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:00.034752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:00.034948 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:00.035125 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:00.035640 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.036875 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.038893 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.039623 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.039771 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:00.040538 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:00.041256 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:00.042133 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:00.042207 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:33:00.042290 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":628,"threads_started":1}
I20260806 22:33:00.062572 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:00.051309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.1.8.124:43607)
0806 22:33:00.051480 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:00.051486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:00.054293 (+ 2807us) server_negotiation.cc:407] Connection header received
0806 22:33:00.054617 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:00.054626 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:00.054748 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:00.054830 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:00.055876 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.057239 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.060127 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.060789 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.060894 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:00.061688 (+ 794us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:00.061833 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:00.061898 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:00.062156 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:33:00.062248 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:33:00.063208 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:00.050778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.1.8.126:46187)
0806 22:33:00.053939 (+ 3161us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:00.053947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:00.055429 (+ 1482us) server_negotiation.cc:407] Connection header received
0806 22:33:00.055612 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:00.055618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:00.055843 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:00.056027 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:00.056857 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.057960 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.060163 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.060794 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.060926 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:00.061683 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:00.062134 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:00.062237 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:00.062802 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 22:33:00.062901 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3021,"thread_start_us":2919,"threads_started":1}
I20260806 22:33:00.067219 4757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:00.054581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.1.8.125:45883)
0806 22:33:00.055138 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:00.055146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:00.060052 (+ 4906us) server_negotiation.cc:407] Connection header received
0806 22:33:00.060357 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:00.060363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:00.060455 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:00.060517 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:00.061457 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.062539 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.065160 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:00.065738 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:00.065918 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:00.066380 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:00.066533 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:00.066603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:00.066775 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:33:00.066875 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402}
I20260806 22:33:00.071381 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4175
I20260806 22:33:00.072544 4273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:00.208062 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4175
I20260806 22:33:00.239625 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4292
I20260806 22:33:00.240811 4384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:00.353037 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4292
I20260806 22:33:00.382076 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4404
I20260806 22:33:00.383302 4512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:00.491422 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4404
2026-08-06T22:33:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5393 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:00 dist-test-slave-7vjv krb5kdc[4795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:00 dist-test-slave-7vjv krb5kdc[4795](info): set up 2 sockets
Aug 06 22:33:00 dist-test-slave-7vjv krb5kdc[4795](info): commencing operation
krb5kdc: starting...
W20260806 22:33:01.844293 1057 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.001s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/test-user.keytab.
Aug 06 22:33:02 dist-test-slave-7vjv krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
Entry for principal kudu/127.1.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab.
I20260806 22:33:02.203229 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:35229
--webserver_interface=127.1.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:02.560495 4814 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:33:02.560882 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:02.561044 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:02.561187 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:02.569079 4814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:02.569171 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:02.569276 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:02.569372 4814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:02.569489 4814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:02.579419 4814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.126.keytab
--principal=kudu/127.1.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.126:35229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:02.580861 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:02.582213 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:02.594303 4826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:02.594388 4827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:02.594399 4829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:02.595249 4814 server_base.cc:1034] running on GCE node
I20260806 22:33:02.596064 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:02.597538 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:02.598762 4814 hybrid_clock.cc:648] HybridClock initialized: now 1786055582598703 us; error 32 us; skew 500 ppm
I20260806 22:33:02.599104 4814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:02 dist-test-slave-7vjv krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:02.602501 4814 init.cc:377] Logged in from keytab as kudu/127.1.8.126@KRBTEST.COM (short username kudu)
I20260806 22:33:02.604044 4814 webserver.cc:460] Webserver started at http://127.1.8.126:33213/ using document root <none> and password file <none>
I20260806 22:33:02.604800 4814 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:02.604930 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:02.605285 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:02.608413 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data/instance:
uuid: "e9a9e297c2594cef96f8c18cc808f57c"
format_stamp: "Formatted at 2026-08-06 22:33:02 on dist-test-slave-7vjv"
server_key: "2cd4feb7dbb8b31b48da7d54ec00d6b7"
server_key_iv: "4660f9a8f23ab34b3e15f0ac3eb61321"
server_key_version: "encryptionkey@0"
I20260806 22:33:02.609232 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance:
uuid: "e9a9e297c2594cef96f8c18cc808f57c"
format_stamp: "Formatted at 2026-08-06 22:33:02 on dist-test-slave-7vjv"
server_key: "2cd4feb7dbb8b31b48da7d54ec00d6b7"
server_key_iv: "4660f9a8f23ab34b3e15f0ac3eb61321"
server_key_version: "encryptionkey@0"
I20260806 22:33:02.614449 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:33:02.618116 4840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:02.619535 4814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:33:02.619755 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal
uuid: "e9a9e297c2594cef96f8c18cc808f57c"
format_stamp: "Formatted at 2026-08-06 22:33:02 on dist-test-slave-7vjv"
server_key: "2cd4feb7dbb8b31b48da7d54ec00d6b7"
server_key_iv: "4660f9a8f23ab34b3e15f0ac3eb61321"
server_key_version: "encryptionkey@0"
I20260806 22:33:02.619994 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:02.653770 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:02.657747 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:02.658044 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:02.676765 4814 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.126:35229
I20260806 22:33:02.676785 4924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.126:35229 every 8 connection(s)
I20260806 22:33:02.678196 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:02.681941 4925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:02.687593 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4814
I20260806 22:33:02.687788 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-0/wal/instance
I20260806 22:33:02.688361 1057 external_mini_cluster.cc:1428] Setting key 06fed49df192993162f0577ec62afc9d
I20260806 22:33:02.689011 4925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:33:02 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:33:02.715109 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.692797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:35229 (local address 127.0.0.1:58654)
0806 22:33:02.693653 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.693705 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:02.696529 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:02.697379 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:02.697388 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:02.698191 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:02.698705 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:02.698731 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:02.700319 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:02.700327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:02.702148 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:02.702165 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:02.703219 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:02.703228 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:02.703460 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:02.704215 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:02.704247 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:02.706901 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:02.711270 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:02.711280 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:02.711289 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:02.711675 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:02.712203 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:02.712206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:02.712208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:02.712353 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:02.713315 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:02.713321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:02.713505 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:33:02.713719 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 22:33:02.714065 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":339,"threads_started":1}
I20260806 22:33:02.715137 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.693864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.1.8.126:35229)
0806 22:33:02.694604 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:02.694610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:02.694651 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:33:02.696911 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:02.696917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:02.697139 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:02.697292 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:02.698913 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:02.700168 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:02.702415 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:02.702974 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:02.703105 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:02.707139 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:02.707151 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:02.707168 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:02.707208 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:02.711137 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:02.711809 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:02.711820 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:02.711826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:02.712032 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:02.712466 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:02.712473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:02.712479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:02.713177 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:02.713378 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:02.713940 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 22:33:02.714242 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":327,"threads_started":1}
I20260806 22:33:02.718365 4925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:33:02.719415 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.719041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55286)
0806 22:33:02.719200 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.719314 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:33:02.719707 4841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
I20260806 22:33:02.721359 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.720994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55296)
0806 22:33:02.721137 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.721221 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:33:02.721558 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:35229: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
Entry for principal kudu/127.1.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab.
I20260806 22:33:02.766855 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:35229
--webserver_interface=127.1.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:02.770879 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } attempt: 1
I20260806 22:33:02.772027 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.771553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55312)
0806 22:33:02.771769 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.771856 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 22:33:02.773072 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.772724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55320)
0806 22:33:02.772868 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.772939 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:33:02.773277 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:35229: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
I20260806 22:33:02.841586 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } attempt: 2
I20260806 22:33:02.842852 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.842357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55328)
0806 22:33:02.842600 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.842708 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:33:02.843912 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.843580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55330)
0806 22:33:02.843711 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.843789 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:33:02.844197 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:35229: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
I20260806 22:33:02.935564 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } attempt: 3
I20260806 22:33:02.936898 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.936435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55338)
0806 22:33:02.936654 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.936745 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 22:33:02.938053 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:02.937668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55346)
0806 22:33:02.937837 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:02.937909 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 22:33:02.938249 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:35229: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
I20260806 22:33:03.081486 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } attempt: 4
I20260806 22:33:03.082804 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.082243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55354)
0806 22:33:03.082583 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.082671 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 22:33:03.083853 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.083498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55368)
0806 22:33:03.083661 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.083737 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:33:03.084028 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.125:35229: Network error: Client connection negotiation failed: client connection to 127.1.8.125:35229: connect: Connection refused (error 111)
W20260806 22:33:03.114243 4935 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:33:03.114610 4935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:03.114738 4935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:03.114872 4935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:03.122231 4935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:03.122346 4935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:03.122488 4935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:03.122574 4935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:03.122682 4935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:03.132542 4935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.125.keytab
--principal=kudu/127.1.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.125:35229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:03.133810 4935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:03.135174 4935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:03.147053 4948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:03.146996 4945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:03.147321 4943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:03.148075 4935 server_base.cc:1034] running on GCE node
I20260806 22:33:03.148937 4935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:03.150276 4935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:03.151522 4935 hybrid_clock.cc:648] HybridClock initialized: now 1786055583151448 us; error 47 us; skew 500 ppm
I20260806 22:33:03.151846 4935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:03.154994 4935 init.cc:377] Logged in from keytab as kudu/127.1.8.125@KRBTEST.COM (short username kudu)
I20260806 22:33:03.156486 4935 webserver.cc:460] Webserver started at http://127.1.8.125:35513/ using document root <none> and password file <none>
I20260806 22:33:03.157151 4935 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:03.157280 4935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:03.157598 4935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:03.160519 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data/instance:
uuid: "b04241c3d4504cda868d76101efaf591"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "3c824e4ebe8d1e46de3a244a366b51e1"
server_key_iv: "930fe2ec04ae19a6d8984081bf01a7bb"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.161298 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal/instance:
uuid: "b04241c3d4504cda868d76101efaf591"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "3c824e4ebe8d1e46de3a244a366b51e1"
server_key_iv: "930fe2ec04ae19a6d8984081bf01a7bb"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.166182 4935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 22:33:03.169644 4960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:03.170888 4935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:33:03.171079 4935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal
uuid: "b04241c3d4504cda868d76101efaf591"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "3c824e4ebe8d1e46de3a244a366b51e1"
server_key_iv: "930fe2ec04ae19a6d8984081bf01a7bb"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.171306 4935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:03.186250 4935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:03.190174 4935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:03.190450 4935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:03.209139 4935 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.125:35229
I20260806 22:33:03.209158 5054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.125:35229 every 8 connection(s)
I20260806 22:33:03.210537 4935 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/data/info.pb
I20260806 22:33:03.212666 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 4935
I20260806 22:33:03.212878 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-1/wal/instance
I20260806 22:33:03.213460 1057 external_mini_cluster.cc:1428] Setting key 16a8646494a7346cf4100e601c417bcb
I20260806 22:33:03.215231 5055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:03.225461 5055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:33:03.248654 5058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.228862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58662 (local address 127.1.8.126:35229)
0806 22:33:03.229382 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.229388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.230417 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 22:33:03.232301 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.232309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.232415 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.232555 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.234020 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.235164 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.236732 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.237503 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.237646 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.241119 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.241127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.241135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.241193 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.245065 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.245837 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.245846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.245853 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.246014 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.246419 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.246427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.246433 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.246859 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.247025 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.247835 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.248130 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
I20260806 22:33:03.249315 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.229388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:35229 (local address 127.0.0.1:58662)
0806 22:33:03.230223 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.230281 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.231967 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.232739 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.232746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.233281 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.233795 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.233812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.235328 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.235332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.236575 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.236588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.237706 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.237711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.237903 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.238599 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.238638 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.240873 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.245236 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.245243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.245256 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.245694 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.246126 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.246129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.246131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.246293 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.246983 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.246989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.247437 (+ 448us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.247663 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.248043 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":307,"threads_started":1}
I20260806 22:33:03.251061 5055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.8.124@KRBTEST.COM" created.
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:33:03.266412 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.251904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55378)
0806 22:33:03.252087 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.252133 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.252230 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.253068 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.253072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.253421 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.253557 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.253565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.255085 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.255091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.256554 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.256573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.257424 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.257434 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.257639 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.258235 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.258262 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.260704 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.263786 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.263793 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.263797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.264253 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.264623 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.264629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.264633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.264768 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.265407 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.265411 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.265550 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.265705 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.265970 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:33:03.266470 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.251965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.1.8.125:35229)
0806 22:33:03.252660 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.252669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.252710 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:33:03.252755 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.252761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.252931 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.253051 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.253703 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.254917 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.256716 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.257243 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.257339 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.260912 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.260923 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.260935 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.260967 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.263658 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.264388 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.264395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.264397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.264508 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.264878 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.264888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.264891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.265249 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.265484 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.265867 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.266068 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":339,"threads_started":1}
I20260806 22:33:03.269043 5055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:33:03.270350 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.269788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40536)
0806 22:33:03.270002 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.270153 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:33:03.270653 4964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.272285 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.271916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40540)
0806 22:33:03.272060 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.272189 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:33:03.272511 5055 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
Entry for principal kudu/127.1.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab.
I20260806 22:33:03.296905 1057 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
/tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:35651
--webserver_interface=127.1.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.8.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:03.321803 5055 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 1
I20260806 22:33:03.323022 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.322529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40554)
0806 22:33:03.322759 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.322846 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:33:03.324251 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.323847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40558)
0806 22:33:03.324009 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.324128 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:33:03.324448 5055 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.383466 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } attempt: 5
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:33:03.392760 5055 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 2
I20260806 22:33:03.393838 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.393296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40570)
0806 22:33:03.393518 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.393628 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:33:03.395349 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.394860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40580)
0806 22:33:03.395064 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.395152 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:33:03.395615 5055 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.398406 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.384417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.1.8.125:35229)
0806 22:33:03.384617 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.384624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.384647 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:33:03.384894 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.384901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.385021 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.385137 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.385970 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.386690 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.387851 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.388381 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.388522 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.392247 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.392252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.392255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.392288 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.395758 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.396566 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.396570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.396572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.396702 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.397144 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.397148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.397150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.397436 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.397549 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.398050 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.398211 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:33:03.398680 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.384306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55388)
0806 22:33:03.384568 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.384590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.384731 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.385180 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.385187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.385611 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.385813 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.385823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.386841 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.386847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.387695 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.387707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.388541 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.388546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.388726 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.389350 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.389367 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.392040 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.395892 (+ 3852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.395899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.395903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.396457 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.396818 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.396825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.396831 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.397025 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.397566 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.397574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.397765 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.397953 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.398318 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 22:33:03.399643 4925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:33:03.400779 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.400402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40586)
0806 22:33:03.400571 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.400643 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:33:03.401795 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.401511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40598)
0806 22:33:03.401635 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.401697 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 22:33:03.402081 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.453344 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 1
I20260806 22:33:03.454582 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.454116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40614)
0806 22:33:03.454312 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.454407 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 22:33:03.455686 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.455336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40616)
0806 22:33:03.455503 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.455578 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 22:33:03.455911 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.486840 5055 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 3
I20260806 22:33:03.488081 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.487582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40630)
0806 22:33:03.487824 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.487911 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 22:33:03.489236 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.488796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40642)
0806 22:33:03.489044 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.489120 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260806 22:33:03.489488 5055 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.524228 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 2
I20260806 22:33:03.525516 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.525018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40648)
0806 22:33:03.525249 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.525343 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:33:03.526587 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.526244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40650)
0806 22:33:03.526396 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.526474 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:33:03.526798 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.632689 5055 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 4
I20260806 22:33:03.633098 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 3
I20260806 22:33:03.634063 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.633438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40656)
0806 22:33:03.633759 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.633861 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 22:33:03.634346 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.633848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40672)
0806 22:33:03.634061 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.634162 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 22:33:03.635118 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.634757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40674)
0806 22:33:03.634892 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.634974 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:33:03.635293 5055 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
I20260806 22:33:03.635667 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.634976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40686)
0806 22:33:03.635476 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.635576 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260806 22:33:03.635866 4925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.8.124:35651: Network error: Client connection negotiation failed: client connection to 127.1.8.124:35651: connect: Connection refused (error 111)
W20260806 22:33:03.652966 5074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:33:03.653307 5074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:03.653414 5074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:03.653548 5074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:03.660939 5074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:03.661015 5074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:03.661136 5074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:03.661223 5074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:03.661336 5074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:03.671181 5074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.8.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.8.126:35229,127.1.8.125:35229,127.1.8.124:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/krb5kdc/kudu_127.1.8.124.keytab
--principal=kudu/127.1.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.8.124:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:03.672525 5074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:03.673820 5074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:03.681565 4916 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.1.8.126:35229
W20260806 22:33:03.686232 5081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:03.686312 5084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:03.686286 5080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:03.687197 5074 server_base.cc:1034] running on GCE node
I20260806 22:33:03.687860 5074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:03.689239 5074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:03.690452 5074 hybrid_clock.cc:648] HybridClock initialized: now 1786055583690370 us; error 55 us; skew 500 ppm
I20260806 22:33:03.690798 5074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:03.694092 5074 init.cc:377] Logged in from keytab as kudu/127.1.8.124@KRBTEST.COM (short username kudu)
I20260806 22:33:03.695572 5074 webserver.cc:460] Webserver started at http://127.1.8.124:41203/ using document root <none> and password file <none>
I20260806 22:33:03.696377 5074 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:03.696522 5074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:03.696885 5074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:03.699923 5074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data/instance:
uuid: "ec9f9a67c68d463eb5cc67e44953144d"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "9417c8c5fcc10a065d1460b366fcfd9a"
server_key_iv: "6bd26f9920a24a32001ad36c8e2dbbf4"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.700827 5074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal/instance:
uuid: "ec9f9a67c68d463eb5cc67e44953144d"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "9417c8c5fcc10a065d1460b366fcfd9a"
server_key_iv: "6bd26f9920a24a32001ad36c8e2dbbf4"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.705996 5074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:33:03.709340 5098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:03.710558 5074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:33:03.710773 5074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data,/tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal
uuid: "ec9f9a67c68d463eb5cc67e44953144d"
format_stamp: "Formatted at 2026-08-06 22:33:03 on dist-test-slave-7vjv"
server_key: "9417c8c5fcc10a065d1460b366fcfd9a"
server_key_iv: "6bd26f9920a24a32001ad36c8e2dbbf4"
server_key_version: "encryptionkey@0"
I20260806 22:33:03.710992 5074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:03.741401 5074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:03.746227 5074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:03.746534 5074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:03.765601 5074 rpc_server.cc:307] RPC server started. Bound to: 127.1.8.124:35651
I20260806 22:33:03.765610 5175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.8.124:35651 every 8 connection(s)
I20260806 22:33:03.766966 5074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/data/info.pb
I20260806 22:33:03.770684 5176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:03.772305 1057 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu as pid 5074
I20260806 22:33:03.772507 1057 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksNPSnJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055191034056-1057-0/minicluster-data/master-2/wal/instance
I20260806 22:33:03.773110 1057 external_mini_cluster.cc:1428] Setting key be3de2efd6eb202c773e4a994cd6d7b0
I20260806 22:33:03.781078 5176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:33:03.796999 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.775011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.1.8.124:35651)
0806 22:33:03.775824 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.775841 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.775895 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:33:03.777852 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.777880 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.778412 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.778861 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.780263 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.782264 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.783510 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.784710 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.784914 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.788866 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.788878 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.788891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.788941 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.792241 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.793042 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.793050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.793058 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.793263 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.793627 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.793633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.793638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.794374 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.794689 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.795047 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.795524 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":340,"threads_started":1}
I20260806 22:33:03.799213 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.783828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.126:35229 (local address 127.0.0.1:58672)
0806 22:33:03.784880 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.784922 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.785070 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.785449 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.785455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.786051 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.786290 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.786298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.787317 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.787324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.788452 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.788464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.789439 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.789444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.789657 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.790290 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.790311 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.792835 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.796736 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.796745 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.796750 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.797213 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.797626 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.797631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.797634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.797804 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.798322 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.798331 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.798516 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.798676 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.798920 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":608,"threads_started":1}
I20260806 22:33:03.799829 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.783818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.1.8.126:35229)
0806 22:33:03.784526 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.784534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.785126 (+ 592us) server_negotiation.cc:407] Connection header received
0806 22:33:03.785198 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.785203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.785274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.785390 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.786443 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.787168 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.788623 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.789247 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.789386 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.793050 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.793057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.793061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.793099 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.796605 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.797368 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.797372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.797374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.797512 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.797917 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.797920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.797922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.798185 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.798300 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.798851 (+ 551us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.799099 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":344,"threads_started":1}
I20260806 22:33:03.801292 5176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.126@KRBTEST.COM
I20260806 22:33:03.813117 4925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 4
I20260806 22:33:03.816963 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.802092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.125:35229 (local address 127.0.0.1:55402)
0806 22:33:03.802405 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.802420 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.802549 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.802869 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.802874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.803335 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.803516 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.803526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.804936 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.804939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.806228 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.806243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.807160 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.807165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.807300 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.807825 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.807840 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.810128 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.813937 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.813943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.813948 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.814485 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.814985 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.814991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.814994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.815205 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.815959 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.815964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.816143 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.816299 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.816594 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:33:03.817327 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.802126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55402 (local address 127.1.8.125:35229)
0806 22:33:03.802310 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.802316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.802503 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:33:03.802671 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.802675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.802737 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.802813 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.803695 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.804785 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.806386 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.806985 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.807123 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.810314 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.810319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.810322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.810357 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.813763 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.814641 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.814647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.814650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.814835 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.815373 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.815379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.815382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.815739 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.815895 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.816419 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.816678 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:33:03.818101 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.800478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.1.8.126:35229)
0806 22:33:03.800903 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.800907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.800919 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:33:03.800966 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.800970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.801081 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.801179 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.802530 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.803611 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.805283 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.805908 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.806051 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.811003 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.811011 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.811017 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.811062 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.814648 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.815542 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.815546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.815548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.815666 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.816051 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.816057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.816060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.816455 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.816743 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.817503 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.817739 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332}
I20260806 22:33:03.818632 5176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.126@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.124@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:33:03.829468 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.813765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40704)
0806 22:33:03.814017 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.814036 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.814190 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.814568 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.814572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.815092 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.815310 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.815323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.816634 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.816639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.818011 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.818034 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.818993 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.818999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.819208 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.820243 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.820268 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.822928 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.826716 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.826721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.826725 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.827084 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.827606 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.827613 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.827617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.827804 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.828500 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.828505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.828667 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.828848 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.829197 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260806 22:33:03.829576 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.813823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.1.8.124:35651)
0806 22:33:03.814000 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.814006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.814116 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:33:03.814353 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.814358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.814433 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.814540 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.815433 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.816472 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.818232 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.818838 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.818979 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.823175 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.823185 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.823190 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.823234 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.826580 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.827204 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.827212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.827218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.827441 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.827940 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.827946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.827950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.828340 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.828512 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.828963 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.829189 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":30976}
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.8.125@KRBTEST.COM
I20260806 22:33:03.835301 4925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: Bootstrap starting.
I20260806 22:33:03.836751 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.819952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40706)
0806 22:33:03.820222 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.820240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.820338 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.820883 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.820887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.821356 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.821552 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.821566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.823038 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.823046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.824706 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.824723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.826003 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.826011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.826227 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.827327 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.827347 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.830298 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.834013 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.834020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.834023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.834498 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.834945 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.834950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.834953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.835119 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.835753 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.835758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.835911 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.836051 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.836393 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 22:33:03.837452 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.819466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.1.8.124:35651)
0806 22:33:03.820268 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.820274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.820298 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:33:03.820458 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.820463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.820545 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.820662 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.821744 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.822896 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.824880 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.825686 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.825806 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.830551 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.830560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.830567 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.830612 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.833852 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.834655 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.834661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.834665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.834810 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.835262 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.835269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.835272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.835614 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.835762 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.836215 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.836571 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":662,"thread_start_us":384,"threads_started":1}
I20260806 22:33:03.839207 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.820212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.1.8.125:35229)
0806 22:33:03.820364 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.820371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.820498 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:33:03.820652 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.820657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.820723 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.820823 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.822086 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.823123 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.825377 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.825792 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.825928 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.832263 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.832268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.832271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.832301 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.835852 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.836729 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.836735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.836739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.836906 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.837405 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.837413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.837417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.837783 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.838043 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.838650 (+ 607us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.838890 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:33:03.839542 4925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:03.841099 4925 log.cc:824] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:03.842888 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: Bootstrap starting.
I20260806 22:33:03.844157 4925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: No bootstrap required, opened a new log
I20260806 22:33:03.846683 5176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:03.847774 5176 log.cc:824] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:03.847884 1057 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:33:03.850693 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: No bootstrap required, opened a new log
I20260806 22:33:03.851291 4879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e9a9e297c2594cef96f8c18cc808f57c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58688
I20260806 22:33:03.855131 4925 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.855511 4925 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a9e297c2594cef96f8c18cc808f57c, State: Initialized, Role: FOLLOWER
I20260806 22:33:03.856019 4925 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.857373 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:03.858111 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:03.858309 5176 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.858623 4925 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:03.858654 5176 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9f9a67c68d463eb5cc67e44953144d, State: Initialized, Role: FOLLOWER
I20260806 22:33:03.858681 4879 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:33:03.858889 4879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:03.859076 5176 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.859987 5188 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:03.860503 5188 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:03.861352 5176 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:03.863165 4879 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.865969 4879 leader_election.cc:288] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [CANDIDATE]: Term 1 election: Requested vote from peers b04241c3d4504cda868d76101efaf591 (127.1.8.125:35229), ec9f9a67c68d463eb5cc67e44953144d (127.1.8.124:35651)
I20260806 22:33:03.866271 5139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9a9e297c2594cef96f8c18cc808f57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ec9f9a67c68d463eb5cc67e44953144d"
I20260806 22:33:03.866803 5139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:03.871021 5139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a9e297c2594cef96f8c18cc808f57c in term 1.
I20260806 22:33:03.871804 4842 leader_election.cc:302] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e9a9e297c2594cef96f8c18cc808f57c, ec9f9a67c68d463eb5cc67e44953144d; no voters:
I20260806 22:33:03.872521 5187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:03.878494 5187 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [term 1 LEADER]: Becoming Leader. State: Replica: e9a9e297c2594cef96f8c18cc808f57c, State: Running, Role: LEADER
I20260806 22:33:03.879303 5187 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:03.907563 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a9e297c2594cef96f8c18cc808f57c. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:03.908293 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: This master's current role is: LEADER
W20260806 22:33:03.911012 4848 tablet.cc:2369] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:03.923523 5237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:03.934523 5055 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } attempt: 5
Aug 06 22:33:03 dist-test-slave-7vjv krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.8.125@KRBTEST.COM for kudu/127.1.8.124@KRBTEST.COM
I20260806 22:33:03.950078 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.882110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.1.8.126:35229)
0806 22:33:03.882300 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.882309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.895644 (+ 13335us) server_negotiation.cc:407] Connection header received
0806 22:33:03.896217 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.896226 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.896339 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.896430 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.901336 (+ 4906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.902607 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.910104 (+ 7497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.911107 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.911239 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.926962 (+ 15723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.926973 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.926980 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.927102 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.932621 (+ 5519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.938931 (+ 6310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.938938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.938941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.939167 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.939920 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.939929 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.939935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.946084 (+ 6149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.948907 (+ 2823us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.949111 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.949435 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:33:03.955706 5237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:03.966813 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.935357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.8.124:35651 (local address 127.0.0.1:40732)
0806 22:33:03.935895 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:03.935922 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:03.936073 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:03.937391 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:03.937400 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:03.938145 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:03.938451 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.938470 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.940597 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.940607 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.942723 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:03.942739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.944668 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.944678 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:03.945036 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.946473 (+ 1437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:03.946510 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:03.950905 (+ 4395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:03.963582 (+ 12677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.963588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.963591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.964042 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.964773 (+ 731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:03.964778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:03.964781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:03.964951 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:03.965723 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:03.965728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:03.965897 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:33:03.966037 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:33:03.966371 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260806 22:33:03.968146 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.936411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.1.8.124:35651)
0806 22:33:03.936725 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.936735 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.936762 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:33:03.936859 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.936868 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.936978 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.937121 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.938722 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.940338 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.942985 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.944141 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.944326 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.951180 (+ 6854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.951191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.951200 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.951258 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.963088 (+ 11830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.964450 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.964455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.964458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.964631 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.965138 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.965143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.965146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.965570 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.967241 (+ 1671us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.967442 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.967676 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 22:33:03.974103 5055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: Bootstrap starting.
I20260806 22:33:03.965662 4814 master_runner.cc:186] Error getting master registration for 127.1.8.125:35229: OK, instance_id { permanent_uuid: "b04241c3d4504cda868d76101efaf591" instance_seqno: 1786055583203715 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:33:03.976358 4814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:33:03.978004 5055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:03.979125 5055 log.cc:824] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:03.980074 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.888883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.1.8.124:35651)
0806 22:33:03.889168 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.889178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.906402 (+ 17224us) server_negotiation.cc:407] Connection header received
0806 22:33:03.906555 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.906561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.906647 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.906726 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.913625 (+ 6899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.915001 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.924334 (+ 9333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.925273 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.925419 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.941567 (+ 16148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.941578 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.941586 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.941641 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.946542 (+ 4901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.968644 (+ 22102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.968654 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.968661 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.968885 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.972641 (+ 3756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.972651 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.972658 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.973228 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.973592 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.979080 (+ 5488us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.979461 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:33:03.983871 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:03.881707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.1.8.125:35229)
0806 22:33:03.881919 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:03.881925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:03.902227 (+ 20302us) server_negotiation.cc:407] Connection header received
0806 22:33:03.902711 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:03.902720 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:03.902810 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:03.902874 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:03.920257 (+ 17383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.921678 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.928852 (+ 7174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:03.929741 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:03.930186 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:03.951234 (+ 21048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:03.951243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:03.951251 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:03.951301 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:03.956476 (+ 5175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.971974 (+ 15498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.971984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.971990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.972286 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:03.976296 (+ 4010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:03.976304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:03.976310 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:03.976921 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:03.977264 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:03.983210 (+ 5946us) server_negotiation.cc:299] Negotiation successful
0806 22:33:03.983474 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:33:03.987398 5139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [term 1 FOLLOWER]: Refusing update from remote peer e9a9e297c2594cef96f8c18cc808f57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:03.988726 5221 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:03.997016 5055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: No bootstrap required, opened a new log
W20260806 22:33:03.998757 5108 tablet.cc:2369] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:33:04.011479 5250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:33:04.011638 5250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:33:04.012360 5055 raft_consensus.cc:348] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:04.012799 5055 raft_consensus.cc:729] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04241c3d4504cda868d76101efaf591, State: Initialized, Role: FOLLOWER
I20260806 22:33:04.013381 5055 consensus_queue.cc:260] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } }
I20260806 22:33:04.017437 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.017910 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:03.992435 5074 master_runner.cc:186] Error getting master registration for 127.1.8.125:35229: OK, instance_id { permanent_uuid: "b04241c3d4504cda868d76101efaf591" instance_seqno: 1786055583203715 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:33:04.018461 5074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:33:04.020949 5055 sys_catalog.cc:564] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:04.021442 5013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:04.020979 5014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9a9e297c2594cef96f8c18cc808f57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b04241c3d4504cda868d76101efaf591"
I20260806 22:33:04.026445 5013 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [term 1 FOLLOWER]: Refusing update from remote peer e9a9e297c2594cef96f8c18cc808f57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:04.030025 5014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a9e297c2594cef96f8c18cc808f57c in term 1.
I20260806 22:33:04.032588 5221 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:04.060355 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a9e297c2594cef96f8c18cc808f57c. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.060729 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:04.080803 5260 mvcc.cc:205] Tried to move back new op lower bound from 7315683672000135168 to 7315683671576428544. Current Snapshot: MvccSnapshot[applied={T|T < 7315683672000135168}]
I20260806 22:33:04.095598 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.096053 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:04.098278 5188 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a9e297c2594cef96f8c18cc808f57c. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.098794 5188 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:04.100394 5237 catalog_manager.cc:1347] Generated new cluster ID: cc25b096ce5a45d2b749e431053b16c4
I20260806 22:33:04.100517 5237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:04.100782 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.101071 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:04.111838 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.112397 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:04.115108 5258 mvcc.cc:205] Tried to move back new op lower bound from 7315683672000135168 to 7315683671576428544. Current Snapshot: MvccSnapshot[applied={T|T < 7315683672000135168}]
I20260806 22:33:04.119481 5188 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a9e297c2594cef96f8c18cc808f57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a9e297c2594cef96f8c18cc808f57c" member_type: VOTER last_known_addr { host: "127.1.8.126" port: 35229 } } peers { permanent_uuid: "b04241c3d4504cda868d76101efaf591" member_type: VOTER last_known_addr { host: "127.1.8.125" port: 35229 } } peers { permanent_uuid: "ec9f9a67c68d463eb5cc67e44953144d" member_type: VOTER last_known_addr { host: "127.1.8.124" port: 35651 } } }
I20260806 22:33:04.119877 5188 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:33:04.145745 5237 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:04.147691 5237 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:04.164391 5284 catalog_manager.cc:1259] Loaded cluster ID: cc25b096ce5a45d2b749e431053b16c4
I20260806 22:33:04.164597 5284 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: loading cluster ID for follower catalog manager: success
I20260806 22:33:04.169189 5284 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: acquiring CA information for follower catalog manager: success
W20260806 22:33:04.170707 5284 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 22:33:04.185948 5237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9a9e297c2594cef96f8c18cc808f57c: Generated new TSK 0
I20260806 22:33:04.187608 5237 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:33:04.232534 5049 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.1.8.125:35229
W20260806 22:33:04.324704 1057 mini_kdc.cc:348] Time spent kdestroy: real 0.119s user 0.000s sys 0.033s
W20260806 22:33:04.402513 5182 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.382352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.1.8.126:35229)
0806 22:33:04.382576 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.382583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.396670 (+ 14087us) server_negotiation.cc:407] Connection header received
0806 22:33:04.400767 (+ 4097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.400774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.400885 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.400971 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.402254 (+ 1283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58702: BlockingRecv error: recv got EOF from 127.0.0.1:58702 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 22:33:04.406064 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.382704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.1.8.125:35229)
0806 22:33:04.382909 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.382916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.403468 (+ 20552us) server_negotiation.cc:407] Connection header received
0806 22:33:04.404457 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.404466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.404588 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.404668 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.405808 (+ 1140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55438: BlockingRecv error: recv got EOF from 127.0.0.1:55438 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 22:33:04.408002 5184 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.389498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.1.8.124:35651)
0806 22:33:04.389714 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.389722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.404018 (+ 14296us) server_negotiation.cc:407] Connection header received
0806 22:33:04.406473 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.406479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.406560 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.406633 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.407735 (+ 1102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40744: BlockingRecv error: recv got EOF from 127.0.0.1:40744 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 22:33:04.455471 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.440515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.1.8.125:35229)
0806 22:33:04.440739 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.440746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.453133 (+ 12387us) server_negotiation.cc:407] Connection header received
0806 22:33:04.453724 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.453731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.453911 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.453992 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.455163 (+ 1171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55446: BlockingRecv error: recv got EOF from 127.0.0.1:55446 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 22:33:04.487855 5054 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.1.8.125:35229: Invalid argument: socket address is ambiguous
W20260806 22:33:04.501446 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.488324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.1.8.125:35229)
0806 22:33:04.488542 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.488549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.499254 (+ 10705us) server_negotiation.cc:407] Connection header received
0806 22:33:04.499825 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.499833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.499952 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.500037 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.501197 (+ 1160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55460: BlockingRecv error: recv got EOF from 127.0.0.1:55460 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 22:33:04.551927 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.538519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.1.8.125:35229)
0806 22:33:04.538728 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.538735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.549764 (+ 11029us) server_negotiation.cc:407] Connection header received
0806 22:33:04.550313 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.550320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.550417 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.550504 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.551650 (+ 1146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55462: BlockingRecv error: recv got EOF from 127.0.0.1:55462 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 22:33:04.604636 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.591104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.1.8.125:35229)
0806 22:33:04.591326 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.591333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.602382 (+ 11049us) server_negotiation.cc:407] Connection header received
0806 22:33:04.602932 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.602971 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.603079 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.603165 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.604342 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55474: BlockingRecv error: recv got EOF from 127.0.0.1:55474 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 22:33:04.648075 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.645976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.1.8.125:35229)
0806 22:33:04.646195 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.646201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.646273 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:33:04.646615 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.646621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.646718 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.646844 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.647796 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55490: BlockingRecv error: recv got EOF from 127.0.0.1:55490 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:33:04.671121 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.668882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55496 (local address 127.1.8.125:35229)
0806 22:33:04.669101 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.669107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.669134 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:33:04.669434 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.669448 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.669552 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.669668 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.670835 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55496: BlockingRecv error: recv got EOF from 127.0.0.1:55496 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 22:33:04.708854 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.706976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.1.8.125:35229)
0806 22:33:04.707144 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.707152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.707272 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:33:04.707563 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.707570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.707664 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.707768 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.708622 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55506: BlockingRecv error: recv got EOF from 127.0.0.1:55506 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 22:33:04.779856 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.777656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.1.8.125:35229)
0806 22:33:04.777923 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.777945 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.778286 (+ 341us) server_negotiation.cc:407] Connection header received
0806 22:33:04.778488 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.778492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.778571 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.778660 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.779628 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55520: BlockingRecv error: recv got EOF from 127.0.0.1:55520 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 22:33:04.913816 5066 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:04.911959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.1.8.125:35229)
0806 22:33:04.912254 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:04.912262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:04.912447 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:33:04.912617 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:04.912622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:04.912696 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:04.912786 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:04.913539 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55534: BlockingRecv error: recv got EOF from 127.0.0.1:55534 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:33:05.013763 5250 catalog_manager.cc:1259] Loaded cluster ID: cc25b096ce5a45d2b749e431053b16c4
I20260806 22:33:05.013909 5250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: loading cluster ID for follower catalog manager: success
I20260806 22:33:05.016635 5250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: acquiring CA information for follower catalog manager: success
I20260806 22:33:05.018481 5250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec9f9a67c68d463eb5cc67e44953144d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:33:05.174094 5284 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b04241c3d4504cda868d76101efaf591: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:33:05.179280 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:05.173325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55536 (local address 127.1.8.125:35229)
0806 22:33:05.173540 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:05.173547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:05.173871 (+ 324us) server_negotiation.cc:407] Connection header received
0806 22:33:05.174077 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:05.174084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:05.174248 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:05.174410 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:05.174869 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.175637 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.177081 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.177488 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.177634 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:05.178134 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:05.178626 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:05.178791 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:05.178906 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:33:05.178996 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:33:05.191527 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:05.186085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.1.8.124:35651)
0806 22:33:05.186527 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:05.186533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:05.186660 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:33:05.186836 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:05.186840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:05.187004 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:05.187159 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:05.187505 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.188296 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.189525 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.189904 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.189993 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:05.190456 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:05.191005 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:05.191126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:05.191231 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:33:05.191296 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":247,"threads_started":1}
I20260806 22:33:05.209290 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:05.197461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.1.8.124:35651)
0806 22:33:05.197700 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:05.197709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:05.203695 (+ 5986us) server_negotiation.cc:407] Connection header received
0806 22:33:05.203941 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:05.203949 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:05.204037 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:05.204138 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:05.204875 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.205524 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.207492 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.207908 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.208061 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:05.208508 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:05.208642 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:05.208727 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:05.208921 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:33:05.209000 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:33:05.210668 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:05.197865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.1.8.125:35229)
0806 22:33:05.198093 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:05.198101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:05.203177 (+ 5076us) server_negotiation.cc:407] Connection header received
0806 22:33:05.203455 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:05.203460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:05.203539 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:05.203603 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:05.204448 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.205537 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.208133 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.208791 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.208878 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:05.209823 (+ 945us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:05.209987 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:05.210051 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:05.210278 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:33:05.210360 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:33:05.211282 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:05.197230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.1.8.126:35229)
0806 22:33:05.203255 (+ 6025us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:05.203261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:05.203278 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:33:05.203400 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:05.203408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:05.203632 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:05.203812 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:05.204621 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.205617 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.208133 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:05.208767 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:05.208883 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:05.209669 (+ 786us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:05.210335 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:05.210437 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:05.210880 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 22:33:05.210975 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5898,"thread_start_us":5800,"threads_started":1}
I20260806 22:33:05.219648 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4814
I20260806 22:33:05.220602 4915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:05.330520 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4814
I20260806 22:33:05.362408 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 4935
I20260806 22:33:05.363694 5047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:05.470546 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 4935
I20260806 22:33:05.500602 1057 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu pid 5074
I20260806 22:33:05.501715 5166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:05.610146 1057 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksNPSnJ/build/asan/bin/kudu with pid 5074
2026-08-06T22:33:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5114 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10508 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (394493 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST